What is Mixed Reality and how do XR headsets work?
More and more XR headsets are coming onto the market. But mixed reality is often misleadingly defined. We set the record straight.
Content
What is Mixed Reality?
In 1994, interface researcher Paul Milgram defined mixed reality as a continuum in which individual immersive technologies such as VR, AR, or video-based AR (passthrough AR) exist.
Mixed Reality according to Milgram and Kishino (1994). | Image: MIXED
In his insightful definition, mixed reality is a state characterized by the degree to which real-world and digital content are blended. Mixed reality can theoretically manifest itself in a device as long as it can handle some form of VR and some form of AR.
How are VR, AR and XR headsets different?
Mixed reality glasses are therefore headsets that can handle VR and AR, as well as all forms of mixing digital and physical reality according to the mixed reality continuum defined by Milgram and Kishino.
With a XR headset, you can either hide the physical reality altogether and replace it with a virtual one (Virtual Reality). Or you reconstruct it digitally in the headset (through passthrough technology, see next section) and enhance it with digital elements (Augmented Reality).
In Virtual Reality headsets, the physical environment appears on opaque displays and is entirely digital, in contrast to augmented reality headsets with translucent optics, such as Hololens 2 or Magic Leap 2. Here, the physical environment remains visible through a piece of glass or plastic and is selectively overlaid by digital elements projected directly into the eye.
The Varjo XR-3 mixed reality headset with front-facing sensors that capture the physical environment, similar to "artificial eyes". There's no direct view of physical reality. | Image: Varjo
The industry often equates mixed reality with pass-through technology and makes an unnecessary distinction between mixed reality and Augmented Reality: while the former enables a stronger interaction between digital and physical reality, the latter simply extends and enhances physical reality, it is said.
But mixed reality headsets and augmented reality headsets do the same thing – just based on a different technology.
The Hololens 2 augmented reality headset uses transparent optics. | Image: Microsoft
The passthrough technology of mixed reality headsets has many advantages, but also disadvantages compared to systems with transparent optics. Since sufficiently good displays for AR headsets do not yet exist (see: State of AR technology), the industry is taking a detour via mixed reality headsets, while research and development on AR displays continues.
How do Mixed Reality headsets work?
Examples of existing mixed reality headsets include Meta Quest Pro, Varjo XR-3 and Lynx R-1, while Apple's first headset will also be a mixed reality device, according to numerous reports.
Mixed reality headsets capture the physical environment using cameras built into the headset. Algorithms then reconstruct a stereoscopic view of the world and display it on opaque screens.
This so-called passthrough technology is not new, and has been used in devices like the Meta Quest 2 for quite some time. However, the image quality is usually quite poor. Even newer mixed reality headsets like Meta Quest Pro render the physical environment far from perfect.
Passthrough looks more impressive with the high-priced professional device Varjo XR-3. The following video shows a user threading a yarn using passthrough view. This example shows how well the technology can work with high-quality sensors and high-resolution displays.
Still, there are big hurdles in advancing passthrough technology, as shown in a recent paper published by Meta.
Note: Links to online stores in articles can be so-called affiliate links. If you buy through this link, MIXED receives a commission from the provider. For you the price does not change.
