Meta Quest 3 is the first headset with built-in upper body tracking
Meta Quest 3 will be the first VR headset to offer accurate upper body tracking without the need for external hardware.
Quest 2 and Quest Pro only track three body points: the position of the headset (head) and controllers (hands). Inverse kinematics (IK) is then used to roughly and often inaccurately estimate the position of the elbows, arms, and torso.
Meta Quest 3, on the other hand, has two side cameras that work with the other sensors and AI algorithms to capture additional body joints such as wrists, elbows, shoulders and torso for much more accurate upper body tracking. Meta calls this new feature Inside-Out Body Tracking, or IOBT. It was announced at Meta Connect 2023.
IOBT opens up new possibilities: For example, users can use smooth locomotion to walk in the direction of their torso, not just their head or controllers, which increases immersion, and lean to the side or over an edge without their avatar taking steps in that direction.
Click on the Youtube video above to see video examples of the technology in action.
AI-generated legs are coming for Quest 2, 3 and Pro
Another innovation that Meta introduced at Meta Connect 2023 is AI-generated legs for avatars, a feature that Meta calls Generative Legs. Much research preceded the implementation of this feature.
Unlike the IOBT, the position of the legs and leg joints is not tracked using sensors, but only estimated, with the estimation supported by the IOBT's data.
However, the Generative Legs also work without the IOBT, using only the three-point tracking of the Quest 2 and Quest Pro. Accordingly, the upcoming AI-generated legs will also be supported by the older headsets (unlike the IOBT).
Thanks to Generative Legs, Quest 3 can accurately translate complex movements such as walking, jumping, ducking and squatting to avatars. However, lifting a knee while standing or crossing the legs while sitting is not supported.
Click on the Youtube video above to see video examples.
Meta's upcoming showcase app Dodge Arcade (see video below) demonstrates the benefits of IOBT and Generative Legs. Players must dodge oncoming balls by ducking, squatting, leaning to the side or contorting their upper body. The new systems work together to create a plausible digital representation of the movements of the entire body.
IOBT and Generative Legs are rolling out to developers in December as new components of Meta's Movement SDK. In the same month, Dodge Arcade will launch in the App Lab.
If you want to learn more about these technologies, I'd recommend watching the whole talk. At the end of the video, Meta also shows upcoming implementations in VR apps such as Supernatural, Swordsman, and Drunken Bar Fight.
Note: Links to online stores in articles can be so-called affiliate links. If you buy through this link, MIXED receives a commission from the provider. For you the price does not change.
