Passthrough API will enable new MR applications on Quest
Meta will soon give Quest developers access to the passthrough cameras, opening up new possibilities for mixed reality.
Expect to see entirely new mixed reality applications on Quest next year. At Connect, Meta announced a new API that will give developers access to the camera's passthrough view for the first time.
The Passthrough API will allow mixed reality apps to more accurately recognize and respond to the environment and individual objects, resulting in more immersive and useful MR applications.
"This will enable all kinds of cutting-edge MR experiences. You've got tracked objects, AI applications, fancy overlays, scene understanding, and so much more", said Meta's Mark Rabkin.
Currently, apps are only allowed to access abstract data about the environment (with the user's permission). This includes a rough 3D mesh of the room and boxes representing individual pieces of furniture (see Space Setup on Quest).
Meta says the Passthrough API will be released in early 2025.
A seemingly sudden turnaround at Meta
Meta has so far been reluctant to allow full camera access. CTO Andrew Bosworth recently explained his position:
"Technology exists inside the context of a society. And the more comfortable society is with the technology, the more you can kind of execute freely and the less comfortable the society is, the more cautious you need to be as you bring that technology to market, because if you're not careful you could actually impede its adoption over time.
So I think we took a pretty conservative stance there. I think everyone probably understands why we took that conservative stance - I stand by that as the right thing to do at the time. [...] if we can unlock that functionality, I think we're all excited about what that represents."
Information about the Passthrough API is scarce at the moment, and we are curious to see what restrictions there will be for developers when the API is released.
Note: Links to online stores in articles can be so-called affiliate links. If you buy through this link, MIXED receives a commission from the provider. For you the price does not change.
