This is what a realistic mixed reality screen looks like
Augmented reality may one day replace physical screens. A tech demo on Meta Quest 3 shows what it could look like.
AR visionary Mark Zuckerberg has been painting this vision for years: One day we'll all be wearing fancy AR glasses that project holograms into the world, replacing objects that don't necessarily have to exist in physical form, including any kind of screen made of atoms.
The company is taking the first step in this direction with Quest 3, which is being marketed as the first affordable mixed reality headset. The device uses sensors to capture the physical environment and digitally recreates the real world on the displays, where it can be manipulated and augmented as desired. For example, by adding virtual screens that can be anchored in the real environment as needed. For example, on the wall where the physical TV is located.
A deceptively real mixed reality screen
This kind of mixed reality is far from ready for everyday use. Quest 3 is still a bulky headset, not a pair of glasses, and the resolution and fidelity of the passthrough view isn't optimal. Then there's the software itself, which lags behind Zuckerberg's vision.
Developer Thomas Ratliff can't do anything about the first two, but with some software tricks he's created a virtual screen that looks a lot more real than current solutions. On Twitter, he shows off a mixed reality screen that supports real-time global illumination and reflections. Note how the screen realistically illuminates nearby surfaces and how the image is reflected on the table.
However, the tech demo has two limitations: first, according to Ratliff, the screen is rendered on a PC, not natively on Meta Quest 3. Second, it's only a video player, not a Nintendo Switch emulator or other app that renders games in real time. So don't be fooled into thinking he's playing the latest Mario game on his Quest 3 in mixed reality.
The tech demo could become an app
Ratliff writes on Twitter that he may take his experiment further and publish it as an app in Meta's App Lab or on Sidequest.
It's also possible that someone will beat him to the punch and create a realistic mixed reality screen that visually interacts with the physical environment. After all, there are already a number of video player apps for Meta Quest 3. The hugely popular Virtual Desktop app, which lets you stream your PC screen to your headset, already supports passthrough. Youtube VR may also add such a feature - or Meta itself. All this is just a matter of time and API support on Meta's part.
Whether realistic virtual screens will make mixed reality headsets like the Quest 3 and the upcoming Apple Vision Pro a breakout success is questionable, but they are an essential feature on their way to becoming the ultimate screen.
Note: Links to online stores in articles can be so-called affiliate links. If you buy through this link, MIXED receives a commission from the provider. For you the price does not change.
