Meta cut an essential hardware feature for Quest Pro
UPDATEIn a Q&A session, Andrew Bosworth, Meta's Chief Technology Officer, comments on the lack of a depth sensor. According to Bosworth, Meta originally included a depth sensor. However, because it added cost and weight without providing enough benefits, Meta eventually decided to remove it. This also allowed Meta to use the existing sensors "more clever," Bosworth said.
"You really want to make sure that every single component is carrying its weight," Bosworth says.
However, the fact that the depth camera is missing from the Meta Quest Pro doesn't mean it won't reappear in a future device. "It's possible that we will integrate depth sensing in the future. It really depends on the scenario."
By their very nature, changes are made right up until the very end when developing complex hardware, Bosworth said.
Meta Quest Pro should have received a depth sensor for better mixed reality, according to Zuckerberg. But the 3D camera is missing in the final product.
Zuckerberg said in May that Project Cambria (the R&D codename for the Quest Pro) will have a depth sensor built in. The depth detection of the 2D cameras on Meta Quest 2 was just a "hack," Zuckerberg said in a conversation with tech magazine Protocol. Cambria's new sensor architecture, which includes the 3D camera, will also bring better hardware support for hand tracking, Zuckerberg said.
In an article on the Oculus Blog, Meta also confirmed that Project Cambria will get a depth sensor.
"Cambria will also feature active depth sensing that uses an IR (Infrared) projector. This will be a significant upgrade over the passive stereo depth sensing currently available on Meta Quest 2, which relies only on 2D cameras for sensing depth," the blog post from late May reads.
It also describes how the sensor works: "Cambria's depth projector creates an invisible IR pattern on surfaces in the room, which helps reconstruct surfaces (especially textureless ones which can't be as easily seen by the headset's cameras) with higher fidelity and accuracy."
What a depth sensor can do for mixed reality
This description sounds suspiciously like a LIDAR sensor, which the premium versions of the iPhone and iPad have and which will reportedly be incorporated into Apple's mixed reality headset.
This summer, Apple demonstrated with the "RoomPlan" API how an iPhone or iPad can quickly and precisely create a 3D plan of the environment thanks to a LIDAR sensor - an important feature for mixed reality technology, which aims at realistic interaction between physical space and digital elements.
Meta Quest 2 requires users to draw in the room and objects by hand. This makes the use of room-based mixed reality very cumbersome.
The depth sensor is missing - Meta takes a stand
Hardware analyst Brad Lynch firmly assumed that Meta Quest Pro will have a depth sensor built in. In early June, he leaked CAD blueprints of a Cambria prototype that show a depth sensor directly above the RGB camera (see article image).
But when the mixed reality headset was unveiled in October, the 3D camera was not visible. UploadVR editor David Heaney, who was allowed to try out the Quest Pro in advance, confirmed via Twitter a little later that the Meta dropped the sensor:
The depth sensor was cut from Cambria some time in the past 5 months - Quest Pro only has 2D cameras.
Meta says it's exploring automatic plane detection, but at launch you'll need to manually mark walls & furniture as with Quest 2.
- David Heaney (@Heaney555) October 11, 2022
Heaney also published a statement from Meta regarding Zuckerberg's earlier statements:
"Meta Quest Pro uses forward facing cameras to measure depth in your environment. This works with 3D reconstruction algorithms to render your physical environment inside your headset via Passthrough. Mark was referring to a different depth sensing system included in a previous prototype version of the device. For context, we’re constantly testing and refining our products before launch and it’s common for hardware changes to happen between versions of the device as we build and develop."
Rumors on why the feature was dropped
Why the 3D camera, which would be so important for mixed reality, was dropped at the last minute is something we can only speculate about.
Brad Lynch previously reported that Meta found problems with broken glass (Twitter link) shortly before production. Another rumor says that Meta found out that body contours can be recognized with the 3D cameras (Twitter link). The company reportedly wanted to avoid a reputation-damaging misuse of the technology and, for this reason, did without the depth sensor.
Looking ahead, whether Meta Quest 3 will get a depth sensor is interesting. In early October, Lynch again leaked CAD blueprints, this time of a presumed Quest 3 prototype. According to the blueprints, Meta relies fully on mixed reality hardware - and a depth sensor.
According to Lynch's sources, Meta has recently been working with an Intel RealSense 3D camera. The problem with Cambria's depth sensor in the final product explains why the company switched to a different 3D camera, Lynch said.
In the following Youtube video, Lynch discusses Meta's omitted depth sensor and analyzes possible reasons.
Note: Links to online stores in articles can be so-called affiliate links. If you buy through this link, MIXED receives a commission from the provider. For you the price does not change.
