Vision Pro Developer Video Reveals Hand-Tracking and Occlusion Capabilities
Get a sneak peek at the hand-tracking and occlusion capabilities of Apple's upcoming Vision Pro headset.
Apple's upcoming Vision Pro headset, set to launch in the coming months, is expected to use hand-tracking as its primary input method. The device's hand-tracking technology is divided into two capabilities: 'hand-tracking' and 'hand-occlusion'. Hand-tracking refers to the 3D model of the hand, its joints, and fingertip positions used for interaction, while hand-occlusion deals with overlaying the user's real hand onto virtual content.
A video posted by a Vision Pro developer using Unity shows the hand-tracking and occlusion capabilities of the device. It reveals that hand-occlusion is quite effective, although there is some clipping around the edges of the hand when surrounded by virtual content and moving quickly. The actual hand-tracking position appears to lag behind the hand-occlusion by about 100–200ms. However, it is difficult to determine the true latency of Vision Pro's hand-tracking, as the video only shows the occlusion-tracking hands for comparison.
Video: Unknown, first reported by Road to VR
No VR Controllers planned for Apple Vision Pro
The use of hand-tracking in VR/AR devices is not new, but the level of precision and realism that Apple's Vision Pro seems to offer is significant. The device uses a 'look and tap' modality, combining hand-tracking with eye-tracking for system-level interactions. Developers can also build applications that allow users to interact directly with applications using their hands. This technology could potentially change the way users interact with VR/AR experiences, making them more immersive and intuitive.
There have been reported production problems and VR controller rejection issues with Apple's Vision Pro. The device will not support VR controllers, instead opting for a 'more elegant solution' using hand- and eye-tracking, says Apple. Only certain Bluetooth keyboards for text input in the Vision Pro and gaming controllers from Sony's Playstation and Microsoft's Xbox consoles are initially supported.
Is hand-tracking enough for VR?
The introduction of hand-tracking and occlusion capabilities in Apple's Vision Pro could revolutionize the VR/AR industry. The ability to interact directly with virtual content using one's hands makes VR/AR experiences more immersive and intuitive. However, the apparent latency issues highlighted in the developer video could be a potential drawback, potentially affecting the user experience. High latency is also an issue with the Quest 3's hand-tracking.
The absence of VR controllers might be considered a disadvantage by some, especially those who are used to traditional VR gaming experiences. However, this decision seems to be in line with Apple's aim to create a sleek, elegant device with minimal accessories. Furthermore, gaming does not seem to be a major focus for Apple, although VR games have already been announced for the Vision Pro. Moreover, the support for certain Bluetooth keyboards and gaming controllers could partially offset this disadvantage.
Note: Links to online stores in articles can be so-called affiliate links. If you buy through this link, MIXED receives a commission from the provider. For you the price does not change.
