Face tracking for Meta Quest at an affordable price may be a while off, according to Meta CTO Andrew Bosworth. He sees mixed reality as more important for a headset breakthrough.
Launched in October, the Meta Quest Pro is the first Meta headset to offer eye and face tracking. Eye tracking enables eye contact in social VR, better graphics thanks to foveated rendering, and new input methods, while face tracking provides more realistic avatars.
Because the XR headset costs around $1,500, the technology will only reach a few people for the time being. This low distribution, in turn, means that studios lack the incentive to include the features in their XR apps.
That could change as soon as gaze tracking and face tracking find their way into the Meta Quest's budget product line and become affordable.
Virtual Reality 2.o will still lack face tracking
However, Meta's chief technology officer rules out the possibility of this happening anytime soon. In his recent Q&A on Instagram, Andrew Bosworth says:
"It's gonna be a little while, it's gonna be years. The reason is simple: All those extra cameras and the compute power required to actually to do the face geometry and the recognition, a lot of trade-offs come with it and one of the main trade-offs is cost."
For him, the essential technical features of Virtual Reality 2.0 include standalone character, mixed reality, and hand tracking.
Bosworth is likely referring to Meta Quest 3, which is due out this year. The leak of a Quest 3 prototype already suggested that Meta is placing a focus on mixed reality rather than eye and face tracking. Zuckerberg recently confirmed that Meta Quest 3 will feature advanced mixed reality.
That the device will forgo eye and face tracking is somewhat surprising, especially since Meta's CEO said a year ago that eye and face tracking would be top priorities for the "next Quest." He said he wanted Meta to build the "most human" VR headset that gives the richest sense of social presence. It's possible he was thinking of Meta Quest Pro.
Why mixed reality is the next big step in development
Bosworth elaborates on why Meta is investing so heavily in mixed reality.
"The virtual reality enthusiast and early adopter community is in one place, but the mass market place is still some place else. I think we underestimate the degree to which virtual reality not just for new people to it but mammals in general having their vision covered not having access to the environment around them is a challenge," Bosworth says.
Mixed reality, he said, is simply better in terms of safety and ease of use. It also allows for entirely different, great experiences, he added (see related story: The best mixed reality experiences for Quest 2 & Quest Pro). Since developers can use reality as the setting for their apps and games, they don't have to create their own worlds.
Bosworth also addresses several other issues.
- He cites the reason for the demise of the popular multiplayer game Echo VR: dwindling user numbers of a few tens of thousands of players. The resources spent on it could be used for other purposes that benefit tens of millions of people, rather than a significantly smaller community of fans. Bosworth is referring here to the studio's as-yet-unannounced next project, Ready at Dawn.
- Bosworth sees potential in a cheaper Quest bundle that ships without a touch controller. For that to happen, however, hand-tracking would first have to improve, and the library of hand-tracking apps would have to get bigger and more impressive.
- Meta's chief technology officer is excited about the possibilities of generative AI, especially in the context of the Metaverse. People could one day use it to have 3D objects created for their own worlds without expertise, which should drive user-generated content.