Meta Quest 2: Hand tracking gets major quality update thanks to AI
Meta brings an AI-enhanced version of hand tracking to Quest 2.
Meta introduced experimental hand tracking in December 2019 for Meta Quest 1. In May 2020, visual hand and finger tracking left the beta phase and became a regular feature.
In April 2021, Meta brought high-frequency hand tracking to Quest 2, which improved gesture recognition and reduced latency. The downsides of this optional feature are that hands shake a bit more and the performance of the XR2 chip is artificially downclocked to avoid overheating, which can negatively impact the performance of VR apps.
Just under a year after this upgrade comes "Hand Tracking 2.0", which replaces high-frequency hand tracking and eliminates its drawbacks. The new hand tracking is only supported by Meta Quest 2.
What Hand Tracking 2.0 brings
Hand Tracking 2.0 is the result of a "new approach to computer vision and machine learning," Meta writes. It improves hand and finger tracking in the following ways:
- The system now provides better continuity, which means that hand tracking is more reliable across the board and interrupts tracking less frequently.
- The system can now recognize hands better, even if they are partially obscured or moving quickly. Hands can cross, clap or give high-fives without the hand tracking getting out of step. A raised thumb is now better detected even when the other fingers are not visible.
- The system now more reliably detects the most important gestures, including pinching, gripping, and poking.
High-quality hand interactions introduced earlier this year benefit from Hand Tracking 2.0, as do gestures implemented by developers.
"Whether your interaction is picking up a virtual game piece, hitting a punching bag, or playing an instrument, this update brings you closer to the seamless replication of natural hand movements in VR," Meta writes on the Oculus Developer Blog.
The following videos show how much hand tracking has improved:
How developers find the new Hand Tracking
Hand Tracking 2.0 is rolling out as an SDK and operating system update starting April 19. It will be some time before you could experience the improved technology in VR apps: Developers must first enable it and release a corresponding update for their VR apps.
The older hand tracking 1.0 will remain the default for the time being and will only be replaced by the new version later this year. From then on, developers will work with Hand Tracking 2.0 by default if they decide to use the feature.
Meta developed and tested the technology with a number of developers and studios that have integrated hand tracking into their VR apps. Developer of 3D puzzle game Cubism Thomas Van Bouwel writes about hand tracking 2.0:
"This update to hand tracking is a big step forward in tracking quality and makes playing Cubism with hands feel a lot more stable and consistent. Previously, Cubism’s hand tracking relied on smoothing the hand data input to produce a stable input method. Furthermore, players needed to be taught not to cross their hands since this negatively affected tracking. This is all improved with the latest hand tracking—which is consistent enough for me to turn off hand smoothing by default."
Other positive feedback comes from the studios behind Unplugged and Hand Physics Lab, which you can read on the Oculus Developer Blog.
Note: Links to online stores in articles can be so-called affiliate links. If you buy through this link, MIXED receives a commission from the provider. For you the price does not change.
