How to Develop a Hand Tracking App for a Jewelry Store

AVRspot has become SmartTek. More info here.

How to Develop a Hand Tracking App for a Jewelry Store

Augmented reality has a huge impact on how customers buy products online. Technology that was previously used for entertainment now helps solve dilemmas with product visualization. And how do you create the best hand tracking app for your needs? In this blog, we’re going to tell you a few success stories of using AR for a virtual try on, and share how we developed a hand tracking app for a jewelry store.

Augmented Reality Online Shopping Apps Success Stories


With an understanding that customers must trust their online purchases, WatchBox, the global e-commerce platform for the selling luxury watches has developed an Augmented Reality feature within the company’s free app, WatchBox.

Augmented reality system allows customers to virtually try on famous brands watches. All watches show up on wrists in their approximate size and shape, allowing shoppers to see how their new watch will look in real life. The system involves printing out and wearing a bracelet, available through the WatchBox website, that works as a marker for the AR app.


FaceCake, the Augmented Reality shopping platform innovator, developed Dangle AR, an iOS mobile app for earrings.

Dangle allows shoppers to find the ideal pair of earrings and then virtually wear it thanks to AR technology. It works in real-time, allowing the customer to feel as though they are just looking into a mirror, exploring the numerous option available to them.


  • It allows shoppers to make sure the jewelry looks great.
  • It helps people to make a decision to buy an expensive item online.
  • Users can get assured how will goods look on them and whether they worth buying.
  • Return rates are reduced in shopping apps that offer virtual try on.
  • Augmented reality provides the personalization customers are looking for.

How We Developed AR Hand Tracking App

Now, the most exciting part. We’ll tell you to step by step what we did to implement an app that detects user’s hand and allows trying on various rings that later customers will be able to buy.

For an AR hand tracking development we opted for OpenCV and color blob algorithm. In computer vision, blob detection methods are aimed at detecting regions in a digital image that differ in properties, such as brightness or color, compared to surrounding regions. In our situation, we’ve applied this method to detect colors of the hand and calculate the median. After the color was defined by an app, it ultimately would cut other colors out.

We took the contour that matches a hand with fingers spread, for the app to detect each finger and place rings. The app only recognizes a hand with fingers spread slightly apart.

The app we implemented works in a similar way to Watch Box and Dangle AR. Except there’s no need for the marker as it is required for Watch Box users.

The user lays his hand on a table, sets his phone over the hand, projects the phone camera on the hand and pickups the ring to try. By detecting each finger potential buyers can easily try on jewelry and choose the best item.

If you have any questions about AR tracking development, contact us — AVRspot experts know how to strengthen your business, implementing efficient and customized augmented reality solutions.