Why Augmented Reality is so freaking hard

Why Augmented Reality is so freaking hard

Tech companies are investing billions in AR headsets, yet the technology is making little headway. Why is that?

They need to be as cool and stylish as Ray-Ban glasses, comfortable to wear, and able to cast lifelike holograms into the environment on command. And they need to be cheap enough that mere mortals can afford them.

These are the sci-fi headsets that companies like Meta, Apple and Microsoft dream about, and whose launch has been postponed time and again because the technology required for them has not yet been invented.

Tech glasses today are either slim and narrow and capable of (almost) no augmented reality, like Ray-Ban Stories, or they are as big and heavy as the Hololens 2 AR headset, which has no use in everyday life. This is the fundamental dilemma of AR headsets.

Sci-fi AR: 22 problems away from reality

Display expert Karl Guttag talks on YouTube about the technical hurdles in developing AR headsets suitable for the masses. Guttag is known as a skeptic of AR headsets: On his blog, he follows industry developments and the AR hype of the last decade with informed criticism.

In a conversation with industry analyst Brad Lynch, he presents a list of 22 design challenges that AR headset manufacturers must overcome if they are to achieve mass adoption and an iPhone moment of augmented reality.

Guttags Liste an Herausforderungen bei der Entwicklung massentauglicher AR-Brillen.

Guttag's long list of design issues. | Image: Karl Guttag

The list provides a good overview of the biggest hurdles, ranging from hard engineering problems (size, weight, power consumption, heat) to drawbacks of current AR displays (image quality, transparency, field of view) to unanswered societal questions (data protection, privacy, social stigma).

AR headsets: A pinch of pragmatism

In addition to the number of technical challenges, it is primarily their relationships to each other that pose major issues for the industry. When a suitable solution has been found for one or more issues, this often has a negative impact on other challenges.


In this context, Guttag speaks of a multidimensional chess game that has to be won. No matter which move you make, you may get closer to one goal, but lose out on another.

The display expert thinks the perfect AR headset is impossible, but he also has good news: that such sci-fi devices may not even be necessary. He counters the exuberant hype and lofty visions with pragmatism: AR headsets have a right to exist if we break away from the maximum requirements and turn to other, smaller markets and applications.

This is how tech glasses could evolve

It doesn't take a wide field of vision and elaborate graphics for AR headsets to be a compelling product. These factors are more important for immersion and gaming rather than everyday use, and are better served by VR headsets, according to Guttag.

Augmented Reality plays to its strengths when it expands the environment in a restrained manner and is downright economical with its resources. It requires less high-end technology to do so. "In VR you provide everything, in AR you want to provide next to nothing," says Guttag, referring to visual stimuli.

Could it be that tech glasses are successfully diversifying into two very different device classes? On the one hand, mixed reality headsets that are particularly powerful and immersive, and on the other hand, sleek AR glasses that are suitable for everyday use and only subtly expand reality, limiting themselves to providing support in everyday life.

The question is how big the market for these types of tech glasses really is. Because none of these devices solves the dilemma mentioned at the beginning.

Sources: Karl Guttags Blog: KGOnTech