Meta's new AR demo for Quest 2 gives a taste of Cambria
The World Beyond demonstrates brand new AR features for Quest 2 and gives us our first taste of advanced passthrough AR with Cambria.
At Connect 2021, Meta introduced the Presence Platform: a set of new interfaces for Quest 2 that include more natural hand and voice interactions as well as augmented reality capabilities.
Some of these interfaces were introduced to developers over the last year and this year, such as the ability to use Quest's passthrough mode or use voice for speech recognition in VR apps.
Yesterday, Meta released a new SDK version. It is an important milestone as it completes all the Presence Platform features announced last year. These can now be fully implemented in VR or AR apps.
Content
New AR features: Digital anchors and spatial understanding
New additions include two AR interfaces: Spatial Anchors and Scene Understanding.
With Spatial Anchors, digital objects can be permanently anchored in physical space. For example, it is possible to hang self-painted artwork and a giant YouTube screen on one's own wall or to set up a permanent workstation with several virtual monitors. Quest 2 remembers the spatial position of the virtual objects so that they appear in the same place when you later put on the VR headset again.
Scene Understanding gives Quest 2 a simple spatial understanding. It can recognize and classify geometry and objects in space. Physical objects can obscure digital elements and vice versa. Developers can use this interface to create AR apps that incorporate furniture and other objects into the experience.
Meta is already working on new Presence Platform interfaces. Some will leverage Project Cambria's unique hardware features, such as face and eye-tracking.
Meta releases AR demo
With the launch of the new SDK, Meta has released The World Beyond AR demo for App Lab. It is intended as an example of what is possible with the new interfaces. Developers have access to the code and can use it to get familiar with the new features.
The first AR apps using Spatial Anchors and Scene Understanding should be released in the next few days. Among them are two demos from VR studios Schell Games (I Expect You To Die, Until You Fall) and Resolution Games (Blaston, Demeo), which are teased in the video above.
The World Beyond: Clear the stage for Oppy
But let's move on to The World Beyond. When you start the AR experience, the first surprise follows: A new feature called "Room Setup" allows you to draw in walls and windows as well as individual objects like sofas and tables. Quest remembers this layout and can thus incorporate physical features and objects into the AR experience.
After setup, the AR experience starts and makes a strange artifact appear from the floor of the room. When you pick it up with your hand, you can shine your open hand into the room and track down a digital creature named Oppy in the cone of light.
With a gesture, you can drag blue balls and throw them around the room, bouncing them realistically off the previously drawn walls. When you play with Oppy and feed him the balls, the creature becomes trusting, so you can pet him with your hands or call him by voice command.
Later, you can make individual room walls disappear, again by gestures. The parallel dimension from which Oppy comes appears in the background: a great augmented reality effect that you have to see with your own eyes.
Video AR with meta-glasses: This is just the beginning
Several interfaces are used in The World Beyond: besides Scene Understanding, Hand Tracking and Voice SDK (voice control) stand out. Despite some great tricks not yet seen on Quest 2: The Presence platform currently only hints at AR potential than realizing it. Convincing AR needs more.
On the one hand, I would like to see a higher-quality RGB passthrough mode, which will probably come in the fall with Project Cambria. For another, I would appreciate it if the Quest 2 could automatically recognize walls and objects. When I move a chair or other piece of furniture, the object in question has to be redrawn each time. The drawn objects are also only rough cuboids, which prevents more realistic masking effects.
The good news is that spatial and object recognition should improve over time. Project Cambria has a special depth sensor that could even detect finer outlines of objects and include them in more detail in the AR experience. So The World Beyond is just the beginning.
Note: Links to online stores in articles can be so-called affiliate links. If you buy through this link, MIXED receives a commission from the provider. For you the price does not change.
