Hand tracking without haptics is a dead end
Hand tracking still has a long way to go. The technology lacks something fundamental for success: haptic feedback.
In August, Meta released the hand tracking demo First Hand, which represents the current state of the art with Meta Quest 2. The technology is impressive, but still is a gimmick despite recent improvements like Hand tracking 2.0.
When I put on my Meta Quest 2, I instinctively reach for the touch controllers. It's faster and somehow feels more direct and satisfying, even when I'm not playing and just navigating the menus.
This is not only due to the sluggishness of the tracking, the dropouts, and lack of precision, in short: the nervous fickleness of the current hand tracking.
I find it unpleasant when I confirm a selection and feel nothing, or worse: virtually pick up a digital object while physically reaching into the air. This doesn't just break immersion. It puts my brain on alert. Hello, something is wrong here.
Hand tracking doesn't need haptic gloves
A recent Twitter discussion about the shortcomings of current hand tracking technology shows that I'm not the only one who thinks this way. After my demo experience, I am more convinced than ever that hand tracking is missing something fundamental, and that is some form of haptics.
I think the unspoken reality is that hand tracking makes sense in some scenarios but completely throwing out tactility & haptics would move us backwards in the medium. The future leans into all of these qualities. https://t.co/A8RHqJt888
- Denny Unger @CloudheadGames? (@DennyCloudhead) September 1, 2022
I don't want haptic gloves for this reason. They are far from ready for the market, as Metas' own research shows, and they are also far too cumbersome to become established in everyday life.
Perhaps much less is already enough. A slight vibration on my wrist when I touch a virtual menu, for example, creates the illusion that I'm doing something physical. Virtual reality proves time and again that the brain is capable of combining different sensory stimuli into a coherent whole, and that we don't necessarily need a lot of technology to do it.
Meta smartwatch could deliver subtle haptics
In this context, I'm thinking of Meta's EMG wristband, which is designed to translate micro gestures into computer commands and provide matching subtle haptic feedback. The haptics side of the wristband goes back to a research project called Tasbi, which debuted in summer 2019.
According to internal studies, the wearable can simulate convincing haptic effects right down to the fingertips. The wristband should even be able to provide haptic support for more complex interactions such as turning a knob, surface texture, and inertia of objects - with minimal technical effort. If Meta manages to miniaturize the technology, it could possibly find a place in a smartwatch.
The wristband is reportedly set to hit the market in the upcoming years. The fact that Meta recently bought Lofelt shows how serious the company is about this venture. The German startup specialized in haptics and made a wristband that turns audio signals into haptic effects.
Apple likely to rely on hand tracking
Apple's upcoming premium headset will appear without a controller, according to reports, and will instead rely solely on hand tracking. If this is indeed the case, I can't imagine Apple doing away with haptic feedback altogether. According to rumors, a ring accessory could be included with the headset for this purpose.
With or without haptics: Hand tracking is far from being ready for the big stage. At least in the form I know it from Meta Quest 2.
But there is hope: Better sensors will increase precision and lower latency, and developers will use software tricks to make hand tracking feel more natural and effortless.
But even then, I might prefer hand tracking only for simple apps and menu navigation, and controllers might stay on the table. Especially for games. Apple and Meta: Convince me otherwise.
Note: Links to online stores in articles can be so-called affiliate links. If you buy through this link, MIXED receives a commission from the provider. For you the price does not change.
