Augmented Reality: Hardware and Definitions
Augmented reality comes in many forms and variations. What devices are there and what are their advantages and disadvantages? An overview.
This article makes an attempt at general categorization. Its purpose is to provide an overview of the most important classes of devices while reflecting the development of the last five years: from AR-enabled mobile devices to AR headsets such as Hololens and Magic Leap to the first passthrough devices coming to market in 2022.
The fifth class of devices, AR-only glasses, describes the goal of large parts of the AR industry and has not yet materialized in this form. The sixth category revolves around tech contact lenses, which are even further away from market maturity than corresponding glasses. It is to be understood as a look into the future of technology.
Other terms or subtypes: Smartphone AR, Tablet AR
Well-known examples: ARKit, ARCore
Advantages: widespread use, mature interfaces
Disadvantages: cumbersome to use, small image detail, no true 3D vision, limited interactivity
Mobile AR is augmented reality using mobile devices such as smartphones and tablets. These use a camera and other sensors as well as computer vision to display digital objects and information context-sensitively in physical space. The digital and physical elements merge on the display of the smartphone or tablet.
The first mature interfaces for Mobile-AR were the Vuforia and Wikitude development kits. In 2017, with the introduction of Apple’s ARKit and the integration of the interface into iOS, Mobile-AR reached the masses. Google followed Apple’s lead in 2018 with ARCore, a solution for Android devices. Other influential mobile-AR interfaces include Spark AR from Meta, Snap AR, and 8th Wall acquired by Niantic.
Since smartphones are ubiquitous, mobile AR is the most common and widely used form of augmented reality. The most popular applications include AR games like Pokémon Go, face filters, Google’s AR search or niche apps for very specific application scenarios like picture frames.
Other terms or subtypes: Smartglasses, video glasses, camera glasses, audio glasses
Well-known examples: Google Glass, North Focals, Ray-Ban Stories, Echo Frames, Snap Spectacles (1-3)
Advantages: comparatively slim form factor
Disadvantages: weak to no AR functionality, narrow field of view
This class includes devices that are based on the form factor of conventional glasses and want to appear as inconspicuous as possible, but only offer rudimentary augmented reality or assisted reality for this reason: They superimpose useful information in the field of view (Google Glass, North Focals) or beam virtual screens into the room (Nreal Air) without taking the physical surroundings into account to any great extent.
Other devices in this class do not have a display but offer alternative smart functions such as cameras and speakers (Ray-Ban Stories) or voice assistance (Echo Frames). The very limited AR functionality is due to technical reasons in all cases: The miniaturization of AR technology is only rudimentary.
The mostly quite narrow field of vision and the associated limited use of the devices in terms of content has prevented wider distribution so far. In some cases, the data glasses are still visually recognisable as such and therefore not socially acceptable. Google Glass flopped because of this and the integrated camera, which triggered a controversy about privacy and data protection.
Other designations or subtypes: Seethrough AR headsets
Well-known examples: Microsoft Hololens (2), Magic Leap (2), Snap Spectacles
Advantages: Advanced AR functionality
Disadvantages: comparatively bulky, narrow field of view, expensive
AR headsets have a different premise than data glasses: Sacrifices in the form factor are accepted to make room for important basic functions of augmented reality like spatial tracking and 3D mapping of the environment.
The devices are correspondingly bulky and should not be confused with conventional glasses, which is why this category is also titled AR headsets instead of AR glasses (see category 5).
The best-known AR headsets are Microsoft Hololens and Magic Leap. The first Hololens appeared in 2016, followed by the Magic Leap One (today: Magic Leap 1) in 2018 and the Hololens 2 in 2019. The Magic Leap 2 will be launched in 2022.
Due to their high price, experimental technology and appearance, the devices are unattractive to consumers. The primary customers are companies, professionals and AR enthusiasts who develop customized applications for the headsets and specific usage scenarios.
Hololens 2 and Magic Leap 2 did bring progress in areas such as form factor, wearing comfort, operation and field of view. However, they did not achieve a technical breakthrough that could make the devices interesting for consumers.
According to rumors, Microsoft has stopped working on Hololens 3. If the rumors are true, Magic Leap would be the only major player still active in the headset market. This could be an indication that the form factor does not have much of a future, anyway, since the manufacturers of these devices want to end up in Category 5 eventually.
Nreal Light and Snap’s Spectacles are interesting hybrids. Both devices look more like conventional sunglasses and offer basic AR functions. However, they do not come close to the technical capabilities of real AR headsets. The form factor is slimmer than a headset like Hololens, but still significantly clunkier than regular glasses.
Video AR headsets
Other terms or subtypes: Passthrough AR headsets, video-viewing AR glasses, VR AR glasses
Examples: Meta Cambria, Lynx R-1, Varjo XR-3
Pros: currently the best AR functionality, wide field of view
Disadvantages: bulky, closed form factor
AR headsets like Hololens and Magic Leap direct light from an image projector into a transparent glass called a waveguide, which then casts the image into the eye. This technology is in its infancy, complex and expensive to manufacture.
Video AR headsets, on the other hand, are based on the same design principle as VR headsets, which makes them easier to manufacture and at the same time reduces costs, as they are based on a largely established technology. They are expected to be the best hardware for advanced augmented reality in the coming years.
Video AR headsets have cameras built into their housings that film the environment and transmit it as a video image to the opaque screens. This creates the illusion of seeing the surrounding world, which can be augmented with digital elements on the displays as desired.
This display technology has great advantages over classic AR headsets with transparent waveguides: it enables a wide field of view comparable to that of VR headsets, realistic masking of physical objects, the display of shadows and darkness and smooth transitions in the mixed reality spectrum, i.e. switching between reality and virtual reality.
Disadvantages of video AR such as latency and a fixed focal plane do not weigh as heavily as the trade-offs of traditional AR headsets (ghosting AR objects, narrow field of view).
That’s why many companies are currently pinning their hopes on video AR headsets. Meta with Cambria and Lynx will launch corresponding devices this year, and similar headsets from Apple, Google, Samsung and Microsoft are rumored to follow in the upcoming years.
While the form factor of VR AR headsets enables advanced AR, it is also the biggest drawback: like VR headsets, the devices are bulky and have limited suitability for everyday use, which is why their use is likely to be largely limited to indoors and designated areas.
Other designations or subtypes: AR Glasses
Advantages: Glasses form factor, excellent AR functionality, wide field of view
Disadvantages: open questions regarding data protection and privacy, technical infrastructure is a major challenge
Stylish and comfortable glasses with AR features that support wearers in their everyday lives, project lifelike holograms into space, and skillfully blend analog and digital reality: that’s the dream the AR industry has been working towards – and it’s been getting barely closer for years. According to Meta’s most important technology researcher, Michael Abrash, this decade will decide whether it ever becomes reality or fails due to physics.
The list of hurdles is long: A proper pair of AR glasses would have to offer immense computing power for the complex tasks involved in AR calculations and still have enough battery power for an entire day, much like our smartphones today.
In addition to the processors, two or more cameras must fit into the frame of the glasses, which the glasses use to orient themselves in the environment and recognize places, objects and people, for example. The glasses must not be too heavy or too warm.
One possible solution to some technical challenges would be 5G streaming: Content comes ready from the cloud, the glasses “only” need to project it into the environment and enable interaction. In this way, some computing load could be outsourced and the glasses could be made slimmer.
But even if these problems were solved, there is still the question of how the public will react to AR glasses that continuously film and analyze their surroundings. That debate has yet to take place. The experience of Google Glass shows that it is likely to be complicated. Meta’s Ray-Ban camera glasses are also already facing privacy criticism.
AR contact lenses
Even more dreams of the future than slim AR glasses are AR contact lenses. Start-ups such as Mojo Vision and Inwith are presenting the first technically impressive prototypes. Their functionality is nevertheless rudimentary, and AR-enabled tech lenses are still much further from market readiness than AR glasses.
Use in the medical environment for certain eye diseases, for example as a contrast enhancer, is more likely for the time being. An external feed such as a pocket computer or cloud streaming is definitely necessary.