Screen Zero: Hololens started as an Xbox headset

Avi Bar-Zeev laid the groundwork for Hololens at Microsoft, then worked for many years at Amazon and Apple on new digital offerings and hardware. Today, he is an investor and advisor to numerous XR startups. How did Hololens get started?

If you were to tell AR enthusiasts from 2015 where VR and AR stand at Microsoft in 2022, they would probably be disappointed. Windows VR was put on the back burner by Microsoft years ago. The Hololens project also seems to be treading water at best.

Most recently, Hololens optics chief Bernard Kress moved to Google, Hololens co-inventor Alex Kipman left Microsoft after more than 21 years with the company, and Rune Jensen, Microsoft's director of AR hardware development who helped build the Hololens team since 2015, recently left for Meta.

"Make it radical!"

In 2010, the world still looked different. Back then, XR was still (or again) at the very beginning of a possible major development - and Microsoft wanted to invest in hardware.

Avi Bar-Zeev was developing innovative product concepts for Microsoft at the time. He was involved in Kinect and telepresence ideas, among others, and had XR headsets for consumers in mind anyway, as Google Glass and Magic Leap were also on the agenda at the time.

Then his innovation team was asked by Microsoft to come up with ideas for the next-generation Xbox after the launch of Kinect. "Make it radical, something that scares us," Xbox executives requested, according to Bar-Zeev.

One screen to replace them all

The puzzle pieces fell into place, and Bar-Zeev and his team began work on "Screen Zero" - a screen that replaces all others. Bar-Zeev left Microsoft about a year after the project began, for reasons that he writes are not worth the space in the article.

But that first year, he says, was the foundation for the Hololens, which was officially unveiled in 2016. There was a particularly high-profile Hololens unveiling back in the summer of 2015 at the E3 gaming trade show - as a next-gen gaming device in tandem with Xbox and Minecraft.

Admittedly, this gaming vision did not come true because the hardware has fundamental weaknesses that Microsoft concealed during the presentation.

"It was groundbreaking. But it’s still not an all-day consumer wearable device. Neither is the Magic Leap 2, or Snap Spectacles, or Varjo, or Quest today," Bar-Zeev writes.

Maximalism vs. minimalism

Bar-Zeev describes the fundamental dilemma of tech glasses or headsets, which he still sees as unresolved: Either you build in a lot of technology for mature AR or VR experiences (maximalism).

Then the devices are clunky and best suited for high-end applications, R&D and industry, Bar-Zeev said. One can hope that the technology will get smaller, but that could take a decade, he said.

"Any kind of strap that people have to slip down and tighten over their heads will limit the diversity of the customer base (consider head size variability, real hair and sensitivity to mussing it up) and make the glasses less likely to be worn. That means the all-day device has to be ultra-light and generally fit like normal glasses," Bar-Zeev writes.

Or you build slim, everyday headsets that sell better but have little application potential (minimalism). Bar-Zeev cites the Walkman as an example of a minimalist device with high sales figures.

Bar-Zeev conceived his Screen Zero project as the happy medium: the headset itself should be minimalist, while the computing power should come from a powerful gaming console - for up to four pairs of glasses simultaneously.

Bar-Zeev's basic idea is reflected today in the Hololens and similar devices: many XR glasses function standalone, but can receive streamed content from a PC or the cloud, with high-quality graphics they couldn't compute on their own.

But even these standalone streaming devices fall far short of the compact form factor of slim glasses.

Headsets should improve everyday tasks

Bar-Zeev believes the minimalist approach has greater market potential than Meta's elaborate devices for photorealistic VR and advocates simple everyday applications.

"Mostly, people want to improve on the kinds of things they do a lot: communicating, navigating, discovering the world around us, understanding and even changing the world in places, buying things, experiencing content and making money through their work efforts. For XR glasses to succeed, they need to do all that significantly better than we can do on smartphones or other approaches," Bar-Zeev writes.

He cites tech glasses that can target the wearer's focus and selectively block light as potential use cases for "proactive and private" communication with the wearer without having to type or formulate a question.

"These glasses can help you remember things, or provide trusted recommendations (vs. push ads) as part of your daily experience," Bar-Zeev writes.

However, he says the technology for this "barely exists today," especially of the minimalist variety, as research has focused on shrinking optical systems while increasing the field of view.

Bar-Zeev also addresses many other technical aspects of past and possible future AR hardware in his Medium article, which is well worth reading.

Note: Links to online stores in articles can be so-called affiliate links. If you buy through this link, MIXED receives a commission from the provider. For you the price does not change.