The history of virtual reality

From the invention of the stereoscope to the metaverse and everything in between, we'll show you the history of virtual reality and how a small 3D image became a virtual world.

The year is 2022. Meta is making virtual reality headsets available to the masses with a consumer focus and affordable pricepoint. Toward the end of June, CEO Mark Zuckerberg displays a number of prototypes that he hopes will someday bring "human eye resolution" displays to the consumer stand-alone market. It seems an odd play from a company whose avatars and virtual worlds are pretty cartoony - but we'd still love to see the tech.

But how did such technical possibilities come about? How did a three-dimensional drawing evolve into a walkable virtual world?

Visionaries with ideas ahead of their time characterize the history of virtual reality. Many ambitious projects failed in the eighties and nineties, for example, due to the technical limitations of their time. Others completely overreached themselves or disillusioned VR enthusiasts with empty promises and misleading marketing. Yes, I'm looking at you "Virtual Boy".

In this article, I'll show you how the forefathers of VR revolutionized the entertainment industry. From the 18th century, VR pioneers did great things. In the 1980s, game console makers failed grandly at virtual worlds. Finally, we'll see how virtual reality finally made the leap into the consumer market in the 2010s.

VR’s forefathers: from panoramic theaters to blue boxes

1788: The panoramic theater changes the entertainment industry

The Irish painter Robert Barker exhibited his first circular painting in Edinburgh in 1788. However, he did not create the associated name until 1792, when he moved his successful exhibition to London. He combined the two Greek words "pan" (all) and "horama" (view) and thus invented the "panorama."

For his 360-degree circular exhibition "The Panorama," he builds a screen 100 meters long and 15 meters high. In the center of the circular structure, he erects a raised, cylindrical platform. With this vantage point, he creates in his visitors the feeling of being right in the middle of the picture. Barker mainly drew landscapes and cities. His son Henry Aston Barker assisted and later continued his father's craft.

The panorama picture sees great success for a long time. However, because viewers need to rotate physically to see the whole image, it causes discomfort in some visitors - motion sickness, if you will. Over time, this art form loses popularity. With the invention of movie theaters at the latest, the panorama theater becomes obsolete.

1838: The stereoscope delivers the first images in 3D

The physicist Sir Charles Wheatstone is fascinated by spatial vision and teaches himself all about it. With his stereoscope, he proves his thesis of binocular perception. The brain processes an object as three-dimensional seen by each eye from a slightly offset viewing angle.

The large device is difficult to use. It consists of a wooden board with two mirrors attached at a 45-degree angle in the center. Pictures are attached to the left and right of them. Wheatstone makes two drawings of the same object, adding minimal shading and changing the angles. When the viewer looks through the device at the mirrors, they provide each eye with a slightly different image.

1849: Improving the Stereoscope

In 1849, Sir David Brewster develops the stereoscope further. He builds a handy box into which one looks through two lenses. A flap changes the position of the images inserted.

Brewster demonstrated his device at the Great Exhibition in Crystal Palace in 1851. "The Queen is amused" and Brewster's stereoscope finds its way into the living rooms of average families. Between 1860 and 1895, one of the highest quality and most successful series of photographs for the stereoscope was produced in Paris with the "Teufeleien".

In the 1920s, the production of stereoscopes is a worldwide industry. Looking through the lenses becomes a glimpse into another world and reaches immense popularity. People gather for communal stereoscope viewings. Images of exotic locations or disasters, such as the great San Francisco earthquake, become coveted collectibles.

The stereoscope also has a controversial side. The new technology is considered disreputable in many places, as some "abuse" it for pornographic images.

1929: The Link Trainer - Flight Simulation from the Blue Box

Shortly after the first successful flights of the Wright brothers at the beginning of the 20th century, the sky belongs to the people - at least for a short time. Not every pilot keeps the elaborately constructed flying vehicles safely in the air. To learn how to handle the flying machines, the first rudimentary flight simulators appear. It takes almost thirty years until organ builder Edward Link revolutionizes flight simulators in 1929.

In a darkened cabin - also known as the "Blue Box" because of its blue color - flight trainees see only the illuminated instruments. The "Link Trainer" teaches pilots to fly their aircraft in poor visibility using only the instruments. An engine-driven unit simulates attitude and turbulence. Turning, tilting and swaying movements provide the necessary realism. The Link Trainer even plays engine sounds.

Link sold its first flight simulators almost exclusively to fairs and amusement parks. It wasn't until the Army Air Corps took over air mail transport in 1934 - and lost twelve pilots and aircraft within a very short time after bad weather flights - that Link's time had come. With the start of World War II, the Link Trainer finally becomes standard equipment at every U.S. flight school.

VR in books - The first stories of virtual worlds

Many technological achievements have one thing in common: they originate in the human imagination. Virtual reality, too, first takes place in people's imaginations.

In 1857, Theophile Gautier first wrote about body swapping in his book "Avatar"(Amazon link). In the book, Octave de Saville, tired of life, has a healer put his consciousness into a different body. While leaving his bleak life behind, he experiences passionate adventures through the eyes of a count.

In "Pygmalion's Spectacles - A Short Story"(Amazon link), Stanley G. Weinbaum describes the principle of VR headset in 1935. In the book, young Dan Berk meets a mysterious professor who lets him try out his latest invention: A miraculous headset. They bring stories to life and transport Berk into an interactive fictional world. He smells, tastes, and even communicates with characters in the story.

VR in its infancy: the cinema of the future and the world’s first VR headset

1962: Sensorama brings cinema to life

Morton Heilig is often referred to as "the father of virtual reality." The studied philosopher, filmmaker, and inventor conceived what was probably the world's first VR arcade in 1955. His "Cinema of the Future" was designed to make films tangible.

Heilig's concept consists of a machine playing a stereoscopic 3D film from a first-person perspective. A vibrating handlebar, a shaking seat, and stereo sound provide the viewer with an immersive experience.

In 1962, Heilig patents his concept as "Sensorama" and goes to market with it. He adds small fans to the experience to simulate airflow. The Sensorama even spreads odors. But, success eludes him. Potential investors are deterred above all by the high production costs. The system is dismissed as too bulky and error-prone for use in arcades.

In 1961, one year before the Sensorama, Heilig patented the "Telesphere Mask". He describes it as a stereoscopic television set for home use. The concept drawing resembles VR headset and, like the Sensorama, is said to include 3D film, stereo headphones, and a smell and wind system. It remains a patent, however. Heilig never builds a prototype.

That same year, the Philco Corporation took up his idea and developed the "Head Sight" video headset. These connect to a camera positioned nearby and display its image. Head movements direct the camera and thus the image on the display. "Head Sight" is being designed for military applications. It is used primarily in war zones to evaluate danger zones from a position of safe cover.

1968: VR headset with weight - The Damocles Sword

In 1965, computer scientist Ivan Sutherland describes his concept of a virtual world. Users are supposed to interact with objects in a simulated environment without having to pay attention to the rules of physics. A device powered by computers provides haptic and visual stimuli. He calls it "The Ultimate Display."

Three years later, he and his student Bob Sproul develop the first computer-connected data glasses. Their design is similar to today's VR headsets, though the experience would more accurately be described as "augmented reality" today.

The device draws a cube with edges about five centimeters long over real objects in space. Similar to Philco's "Head Sight," the image and head movements are synchronized. Two cathode ray tubes generate the virtual image and project it directly onto the eyes via mirrors.

Sutherland's VR device gets the nickname "The Sword of Damocles" because of its weight. The design is so heavy that he has to mount it to the ceiling. Without this mount, it cannot be carried. Sutherland later works with his company, Evans & Sutherland, on display systems for flight simulators. At the Proto Awards 2015, the jury gives him the "Special Founder Award."

VR in the 80s: The first hot phase of virtual reality begins

1980: Eric Howlett’s LEEP system convinces NASA

It takes a while for VR to reappear on the scene. It is not until the technological advances of the 1980s and the interest of the U.S. National Aeronautics and Space Administration (NASA) that Virtual Reality moves forward.

In 1980, Eric Howlett develops a stereoscopic photography system called the "LEEP Panoramic Stereo Photography System." It consists of a camera for wide-angle photography and a viewing device that achieves a field of view of up to 140 degrees.

He presents it for the first time at the Photographic Society of America conference. His system arouses general interest, but neither Kodak nor Polaroid believes in the project.

Howlett tried to produce and distribute it himself but initially failed due to the complex and costly implementation. It wasn't until a NASA engineer named McGreevy learned of Howlett's invention in the mid-1980s that the LEEP system took off: NASA and later VPL Research become its biggest customers.

1981: Thomas Zimmerman conceives the first data glove

MIT graduate and musician Thomas G. Zimmerman works with a college friend on an idea for virtual orchestras. Zimmerman devises a concept to measure finger bending while playing virtual musical instruments.

The first prototype is based on a gardening glove. The finger bending is measured by light. To do this, Zimmerman attaches a hollow tube to each finger, which he equips with an LED light at one end and a phototransistor at the other. The more a finger bends, the less light passes through the tubes. The strength or weakness of the light beam thus measures the degree of bending of the finger.

In 1982, he files a patent for his optical flex sensor. He programs various applications to read the light signal from his glove. In the process, he experiments with sign language, among other things. To expand his programming skills, he even goes back to school.

In his programming class, he meets Nancy Mayer, the wife of ATARI co-founder Steve Mayer. Mayer is the head of a new ATARI research lab that is looking into the future of digital entertainment media, including electronic music.

Zimmerman starts working in ATARI's research lab. He presents his patent there and ATARI offers him $10,000 for the rights. Zimmerman declines. The video game crisis of 1983 forces ATARI to close the research lab. Zimmerman works on voice synthesizers from then on. His optical flex sensor disappears in the drawer (for the time being).

1984: Jaron Lanier and Thomas Zimmerman found VPL Research

When Jaron Lanier learns about Ivan Sutherland's "Sword of Damocles" as a child, he is immediately fascinated. Lanier is enthralled by virtual worlds and working with computers. At the age of fifteen, he is already taking math and science courses at New Mexico State University.

He combines his fascination for computers with his passion for music and begins programming sound for video games. He later develops the innovative music game "Moondust," which lands him a job at ATARI.

ATARI releases Moondust in 1983 for the Commodore 64, but Lanier also falls victim to the video game crisis and loses his job. He uses licensing income from Moondust sales to work on a visual programming language.

Instead of code, he uses characters, symbols, and sounds. Lanier wants to make complex programming accessible to non-computer scientists. The display sizes of common monitors slow Lanier down in his project. They are simply too small to display his visual programming language.

Lanier and Zimmerman meet at a concert at Stanford University. The two talk about Zimmerman's optical flex sensor, which Lanier identifies as the ideal input device for his visual programming language.

He wants to move the virtual objects of his programming language directly with Zimmerman's data gloves, bypassing the restrictions of a conventional PC. Together, Lanier and Zimmerman found VPL Research in 1984. In the years to come, the two work with a team of programmers and engineers on the technical implementation of their visions.

1985: VR pioneers VPL develop the “DataGlove

Visionary Lanier wants to make data tangible and immersion in a digital world possible. He uses the term "virtual reality" for the first time. In the eighties, VPL Research pioneered innovative developments for today's VR.

Lanier and his team further develop his programming language under the name "Body Electric" for use in VR applications. With the real-time 3D rendering engine "Isaac," Lanier creates virtual avatars for the first time.

Ultrasonic tracking enables VPL's data glove - now named "Z-Glove" - to provide three-dimensional hand position recognition. Speakers attached to the glove send out sound waves to three different receivers. Based on the signal's arrival time, the receivers calculate the glove's position and posture. VPL engineer Young Harvill further develops Zimmerman's glove.

The "Data Glove" now measures finger movements with fiber optic cables and uses magnetic sensors for tracking. Harvill's version will be significantly more expensive but achieves significant improvements in accuracy and performance. VPL sees great potential in the technology and is focusing on optimizing the Data Glove as an input device. One of the first interested parties is NASA.

1986: VPL Research and NASA - a fruitful collaboration

NASA's Ames Aerospace Human Factors Research Division is working on experimental human-computer interaction systems. In 1985, an early concept, the "Virtual Environment Display System", was developed. This first prototype is a wide-angle display system mounted on a motorcycle helmet. The sensors for motion control are also attached to it.

In a computer-generated environment, rocket engineers, for example, view their constructs in three-dimensional space. The user's position, voice, and gestures influence objects in the virtual world. The converted motorcycle helmet offers a 120-degree field of vision and is being further developed in cooperation with VPL Research. The collaboration begins in 1986.

The space agency orders VPL's "Data Glove" as an input device for its own VR system. The "VIEW" is a further development of the motorcycle helmet prototype and is based on Eric Howlett's wide-angle "LEEP" system. NASA sees VPL's "Data Glove" as the ideal input device for the VIEW. It easily controls virtual simulations or even robots in real-time. From now on, NASA and VPL conduct joint research on the promising technology.

1987: VPL Research develops the first commercial VR headset

Within a short time, VPL develops its data glove into a data suit - the "DataSuit". This is used primarily in films with motion capture. Based on the knowledge gained from its collaboration with NASA and its VIEW system, VPL develops the world's first commercial VR headset in 1987.

The "Eye-Phone HRX", used primarily within research, costs a hefty $49,000. The further-developed "DataGlove" serves as the input device for around $9,000.

However, the system struggles with technological limitations and high costs. With five to six frames per second, the EyePhone is even far behind the 30 FPS of the common eighties TVs.

The price for a complete set, including the necessary computers, is $250,000 - clearly too high to reach the consumer market and thus not lucrative for investors. VPL Research files for bankruptcy for the first time in 1990.

1989: Nintendo releases the PowerGlove for the NES

Abrams/Gentile Entertainment (AGE) develops toy series for manufacturers such as Hasbro in the 1980s. The design studio sees an interesting market in 3D representation and virtual reality. AGE equips its action figures for the animated series "Visionaries" with 3D holograms.

For his next project, Christopher Gentile, the younger brother of AGE's founders, is designing a 3D gaming system. He wants to do away with traditional input devices like joysticks and gamepads for a more immersive gaming experience. He quickly comes across VPL's "DataGlove" while searching for alternatives.

AGE and VPL agree on a licensing deal. This allows AGE to optimize and distribute the "DataGlove" for video game systems. However, the home console market is dominated by Nintendo at this time and AGE cannot find a partner willing to compete.

Gentile turns around and tries to modify the "DataGlove" as an input device for Nintendo's NES consoles. Using a Macintosh, he manages to control games on the NES with the DataGlove. He approaches Mattel and convinces the toy giant of his "Power Glove".

Hasty market launch and abrupt end of the PowerGlove

Mattel wants to dominate the 1989 Christmas shopping season with Gentile's idea. The $9,000 "DataGlove" is slimmed down to a game controller in just five months. VPL's expensive fiber optic and magnetic tracking sensors are replaced with lower-cost alternatives.

Among other modifications, Mattel is again using the considerably cheaper ultrasonic tracking used in VPL's "Z-Glove". Mattel sells the slimmed-down "DataGlove" for about $80. The manufacturing costs amount to just $23.

Nintendo is skeptical at first and believes that setting up the controller is too cumbersome for children. The Japanese licensed the "Power Glove" after long negotiations and launched it on the market in 1989, just in time for Christmas.

However, the hasty market launch caused the "Power Glove" to fail. Only one game optimized for the data glove is released by the release date of the Power Glove. Another is not released until October of the following year. The controls of already released NES games are wonky or don't work at all. In October 1990 - the release month of the second Power Glove game - Nintendo officially discontinues the Power Glove.

VR hype in the 90s: VR arcades and legendary console flops

1990: Virtuality - Rise of the VR arcades

In 1987, Dr. Johnathan Waldern founds W-Industries together with Al Humbrich, Richard Holmes, and Terry Rowley. Over the next few years, the company develops several VR headset prototypes based on an Amiga 2000 computer.

W-Industries initially plans to use virtual reality as a development tool for computer-aided design. Waldern and his team notice the great potential of VR games and change course. In 1989, W-Industries wins the British Technology Group's "Best Emerging Technology Award" for one of its prototypes.

The award attracts investors. Just one year later, W-Industries presents its first VR arcade, model number "1000SU," at the Computer Graphics 90 Exhibition. The VR demo of the first stand-up VR arcade is an early version of the later "Dactyl Nightmare" - which has a brief resurgence on the Oculus Store as "Polygon Nightmare".

1991: Virtuality unveils the 1000CS and 1000SD VR arcades.

In March 1991, things get serious: W-Industries presents the market-ready Virtuality 1000CS (CS = Cyber Space) and 1000SD (SD = Sit Down) VR-Arcades. The system is based on an AMIGA 3000 with 4 MB RAM and a Dual TMS 32 bit graphics card.

The LC displays of the VR headset reach 20 FPS and a resolution of 276 x 372 pixels. The headset offers a 65-degree field-of-view, but weighs a hefty two pounds. LCDs are mounted on the side and are reflected by mirrors onto the lenses - very similar to the principle of Charles Wheatstone's stereoscope.

A built-in microphone and quadrophonic sound (four-channel) stereo enable conversations between players for the first time in multiplayer. The player sits in the 1000SU unit like in a bumper car. W-Industries even gives the scooters a safety door. The intent is - no joke - to protect men from kicks in the privates and women from glances up their skirts while they use the units.

The setup of the 1000CS unit is complicated and almost impossible alone. The player stands on a platform surrounded by a large plastic ring. This ensures that players don't wander around unintentionally and simultaneously sets up a three-dimensional magnetic field for tracking.

The corresponding receivers are built into the VR headset and determine position and viewing direction. Before an assistant puts the large VR headset on the user, a bulky hardware belt is strapped on.

A holster is attached to it, in which the pistol-shaped controller is inserted. Two thick cables lead from the large VR headset to the hardware belt and from there to the computer.

Shortly after the release of the automatons, Virtuality releases the "Touchglove" as an input device. The haptic glove has a total of twelve small air cushions in the fingers and palms. An air pump inflates the cushions slightly when you place your fingers around a virtual object, creating haptic feedback.

1993: SEGA VR - Invention and Hype

Sega also makes its first 3D walking attempts back in the eighties. With the "SegaScope 3D Glasses", the Sonic makers bring 3D to their master system in 1987. The 3D shutter glasses work similarly to the headset of today's 3D TVs and create a simple 3D depth effect.

For the Genesis console, Sega plans to offer a more immersive gaming experience and enter the virtual reality market. Among other things, SEGA is offered a technique for this that uses red LED light to create three-dimensional objects.

Not very convinced by this approach, Sega rejects licensing - in contrast to competitor Nintendo. Sega wants to offer an appealing technology and convince visually at the same time. The company hires IDEO, the design team behind the Apple Lisa mouse.

The result is a futuristic look inspired by Star Trek's Geordie LaForge, Knight Rider and Robocop. Sega plans to offer its VR headset for $200, but the motion sensors needed for tracking cost close to $50,000 at the time.

At the same time, Mark Pesce was developing a low-cost alternative for his company Ono-Sendai - the name might sound familiar to Neuromancer readers. According to Pesce, the motion sensor costs just one US dollar - the ideal solution for Sega.

In 1992, the two companies began working together. Sega invested in Pesce's research and licensed his sensor for SEGA VR. The VR headset is supposed to generate stereoscopic 3D. Two LC displays provide resolutions of 320 x 240 and 256 x 320 pixels. An integrated stereo sound headset is part of SEGA VR, as is the adjustable headband.

SEGA outsources much of the hardware, as we know from the PlayStation VR's external processor unit. This reduces the weight of the headset and makes it more comfortable. The standard SEGA Genesis controllers are used as input devices. After many pre-announcements in the media, SEGA finally unveils its VR headset at CES 1993, hosted by MTV VJ Allen Hunter. SEGA VR is scheduled for release in December 1993.

The Console Fails

Previews in console magazines, commercials, and even sweepstakes push the release. But there is no concrete release date until the fall, and suddenly it gets quiet about SEGA VR. Officially, the experience is too realistic and immersive. Players would run around uncontrollably and hurt themselves - a risk that SEGA does not want to take. In fact, SEGA has other problems with its headset, which is officially confirmed in the fall of 1993.

Tests at the Stanford Research Institute reveal that SEGA VR causes serious motion sickness or severe headaches in most users. The Nintendo competitor first postpones the release and then finally pulls the plug in 1994. SEGA VR is never released.

1994: Virtuality - 2000CS, Project Elysium and the ATARI disaster

W-Industries continues to develop their own VR arcades. The new version of Virtuality arcades (2000CS) is based on an IBM PC. An Intel 486DX33 processor, 8 MB RAM and 16 MB video RAM provide the necessary graphics power.

The significantly slimmer VR headset weighs less than a pound and a half and can be adjusted for individual eye distance for the first time. Glasses wearers can look forward to a focus correction for each eye. The resolution of the displays is reduced to 255 x 378 pixels. The field of view also shrinks to 60 degrees in the 2000CS. Due to the generally decreasing hardware prices and a cost-saving composition of the components, the new VR arcades will be significantly cheaper. W-Industries charges about $1,200 for a complete system.

In cooperation with IBM, W-Industries developed "Project Elysium" in 1994. The PC-VR system is tailored to business customers in the fields of architecture, medicine, and education. The increasingly popular home consoles slowly diminish interest in arcades - and W-Industries feels this too. The new generation of VR arcades is not the success it had hoped for, despite its cheaper price.

Now renamed Virtuality Group, the company is trying to place its VR headsets in the home consumer market. Cooperation with ATARI is supposed to bring Virtuality into the living rooms, but fails all along the line. At CES 1995, Virtuality presents the joint project together with ATARI. The VR headset for the ATARI Jaguar is to be released in the same year.

Jaguar VR relies on infrared tracking, but the signal is not stable. Connection dropouts and the low resolution lead to motion sickness and poor playability. Virtuality continues to invest in the development of the headset, but after ATARI merges with JTS, the project is scrapped. 1997 Virtuality Group files for bankruptcy.

1995: Virtual Boy - Nintendo's fake VR flop

After the successful 16-bit era, the console market is about to enter a new generation. The SEGA Saturn and the Sony PlayStation are soon to be released.

With Nintendo, of all companies, the market leader of recent years is in danger of missing the launch. "Project Reality" - later known as Nintendo 64 - will not be ready for the market until 1996. As a result, Nintendo is forced to release a project that is actually not yet ready for the market - the Virtual Boy. The US company "Reflection Technology" approaches Nintendo's competitor SEGA in 1991 with a 3D system - and is rebuffed.

However, Nintendo sees a unique selling point in the technology - and the chance to create a completely new gaming experience. Reflection Technology's system basically consists of two LED screens that each cast their images sideways through a lens onto a mirror. Both mirrors are positioned at a 45-degree angle to each eye, creating the 3D effect.

In 1992, Nintendo acquired the exclusive worldwide rights to Reflection Technology's 3D system - for a whopping $5M. Nintendo discards plans to implement the system in a VR headset. Motion tracking is also canceled. Instead, Nintendo puts the Virtual Boy on a stand. The final implementation no longer has much to do with Virtual Reality. Using a gamepad, the player controls red and black Game Boy games with a slight 3D depth effect. The decision for this color scheme is meant to save costs.

The result is nausea and headaches. Since most players have to slouch and lean forward to the lenses, a stiff neck also adds to the list of complaints. The Virtual Boy flops in Japan and the USA. In Europe, it is not even launched on the market. After only eight months and around 800,000 units sold, Nintendo takes the 3D frame off the market.

90-VR: PC-VR with VFX1 and NASA Astronaut Training

NASA remains faithful to VR in the nineties. In 1991, it establishes its own "VR Lab", a research and development facility for virtual reality. Starting in 1994, astronauts are trained in the use of the "Extravehicular Activity Suit," or "EVA" for short, on displays with a resolution of 1,280 x 1,024 pixels.

Besides Nintendo, SEGA and ATARI, VictorMaxx is also trying to get in on the VR hype. Compatible with SNES and SEGA Genesis, the "Stuntmaster" however only brings the pixel games of the consoles to the LC display of a headset. No trace of real VR.

The successor "CyberMaxx" also brings this questionable experience to the home PC. Virtual IO's "i-Glasses" can be connected to the VHS recorder in addition to the PC. Movies are thus transferred to the displays of the VR headset.

In 1995, Forte Technologies releases what is probably the most powerful PC VR headset of its time. The "VFX1" brings a resolution of 263 x 230 pixels to each eye and provides a diagonal 45-degree field of view.

Ultimately, however, all VR headsets for the consumer market share the same fate. Processors from home computers and consoles are simply not powerful enough in the nineties. Frame rates are too low and high latencies lead to nausea, headaches, and eye pain. Compelling VR experiences remain a promise for now.

VR in the 2010s: Oculus, HTC, Valve and Sony launch new VR era

2012: Palmer Luckey founds Oculus VR

Palmer Luckey is about 16 years old when he discovers his fascination with virtual reality. However, the young tech hobbyist is dissatisfied with the way virtual worlds have been implemented so far. He collects old VR headset, learns all about their technology, and modifies the devices.

In 2011, Luckey enrolled in the Institute for Creative Technologies at the University of Southern California to develop his own prototype. He shares his experiences on tech forums like ModRetro and MTBS3d.com and quickly earns a reputation as a VR expert.

In 2012, he comes into contact with John Carmack, who is also very interested in VR. Carmack is a programmer and game developer at id Software. He is one of the best-known names in the gaming industry and is responsible for cult games like Doom and Wolfenstein 3D.

Luckey has now developed his sixth prototype. At E3 2012, Carmack shows a customized VR version of Doom 3, which he demonstrates on Luckey's VR headset. Shortly thereafter, Luckey founded the company Oculus VR together with Brendan Iribe, Michael Antonov, Nate Mitchell and Andrew Scott Reisse.

With a crowdfunding campaign on Kickstarter, the startup generates more than $2.4 million within a few weeks. The company reaches the original funding goal of $250,000 after just four hours.

With the Oculus Rift, Oculus pursues a VR headset for gamers that offers technically high-quality VR at a price of less than $300. In early 2013, Oculus VR will ship the first developer kits of the Rift to developer studios. John Carmack moves from id Software to Oculus VR and takes over the position of Chief Technology Officer.

2014: Facebook buys Oculus, Sony announces a VR headset, and Valve and HTC cooperate

Sony releases the first Move motion controllers for the PlayStation 3 in 2010. The Japanese start experimenting with virtual reality shortly after. Early on, Sony relies on optical tracking from a light source and attaches the glowing motion controllers to Sony HMZ T1 3D video glasses and later prototypes to track their movements.

In 2012, Valve also begins research on VR headset. The company behind the successful software distribution platform Steam focuses mainly on tracking. The first VR headset prototype includes a camera. The camera tracks the position of the headset and controller using various fixed points set in space.

About a year later, Valve uses lasers for the first time in its tracking system. Valve is not interested in launching its own VR headset and aims for collaboration with Oculus. However, this is only short-lived.

The progress and potential of Oculus VR does not go unnoticed. In 2014, Facebook takes over the young company for about $3B. The takeover by Facebook angers individuals at Valve so much that the cooperation ends. Valve finds a new hardware partner in HTC.

A short time later, Oculus releases the Develompent Kit 2 with improvements in resolution and head tracking. In the same month as the Oculus takeover, Sony boss Shuhei Yoshida officially announces his own VR headset. However, the VR headset presented under the working title "Project Morpheus" does not yet have a release date.

2015: HTC & Valve announce Vive, Microsoft moves towards augmented reality with HoloLens

In January 2015, Microsoft causes a surprise during the presentation of the new Windows 10 operating system. The software company announces a holographic headset called "HoloLens". With the associated operating system "Microsoft Holographic", Windows 10 is to become the world's first holographic computer platform.

Microsoft HoloLens is a wireless AR headset that holographically inserts virtual objects into the wearer's real-world environment. With a 3D sensor, it is said to recognize hand movements and does not require an external computer. With that, it's clear that Microsoft itself is leading the way toward augmented reality.

Meanwhile, Valve finds a partner in Taiwan. HTC develops and designs a VR headset based on Valve's external tracking system "Lighthouse". Valve and HTC finally announce the "HTC Vive" at the Mobile World Congress.

2016: The beginning of modern virtual reality

With Facebook and Oculus, HTC and Valve, as well as Sony, the big players in virtual reality are set. Redmond concentrates on augmented reality with the HoloLens. However, Windows Holographic, which was announced for 2017, is now supposed to become the standard OS for mixed reality and thus also support VR.

Originally announced for a fall 2015 release, Valve and HTC are moving the Vive to 2016, making it a spring 2016 celebration for VR enthusiasts. The two PC VR headset HTC Vive and Oculus Rift release within just one week between March 28 and April 5, 2016.

Sony, on the other hand, is focusing on the holiday shopping season, releasing its now renamed PlayStation VR console VR headset on October 13, 2016. Also in October, Microsoft announces new VR headset for Windows. Apps for Windows Holographic are to work on a range of devices, explains Windows boss Terry Myerson.

HP, Dell, Lenovo, Asus, and Acer work on VR headsets for co-marketing with Microsoft. All require wires and handle position tracking with an integrated camera system. That makes installation easier.

Smartphone VR with Google and Samsung

Google and Samsung are trying to make virtual reality accessible to everyone and link it to a technology that is in almost every pocket - the smartphone. The phone simply slides into a holder with lenses strapped in front of the face like VR headset.

In cooperation with Oculus VR, Samsung released the first commercial variant of Samsung Gear VR in November 2015. Samsung produces the smartphone VR headset optimized for Galaxy smartphones. Oculus VR provides both the necessary tracking know-how and its Oculus ecosystem. Samsung thus brings the best smartphone VR experience to date to the market.

A year earlier, Google's cardboard model, "Cardboard", had already appeared. This is a particularly inexpensive entry into virtual reality, but it only offers a rudimentary VR experience at best. Google later followed up with Daydream View, a better smartphone headset, but smartphone VR finally failed in 2018 at the latest. The devices do not achieve significant demand. Google as well as Samsung and Oculus finally drop smartphone VR in 2019 after the lack of success.

2017: VR headsets sell worse than expected and Oculus founder Palmer Luckey exits from Facebook

Three years after Facebook acquired Oculus VR, the inventor of Oculus Rift is leaving the company. After it became known that the 24-year-old company founder may have financially supported anti-Hilliary Clinton campaigns in the American election year, he becomes persona non grata at Facebook.

An apologetic statement heralds his slow disappearance from the public eye. At first, Luckey barely attends PR appointments. March 31, 2017, is finally his last day of work at Facebook. At first, it is unclear whether Luckey is leaving Facebook voluntarily. Later, it becomes clear that Luckey's departure from Facebook was a sacking.

The second big VR hype has meanwhile given way to disillusionment. Virtual reality is too expensive, displays have too low a resolution, and cables are a nuisance. The PC gaming community also complains about the lack of mature games. The technology does not live up to unrealistic expectations and the sales figures are far behind analysts' forecasts even after the Christmas business.

Nevertheless, the Chinese manufacturer Pimax generates almost $4M for its 8K and 5K VR headset via Kickstarter campaign - more than ten times the originally targeted sum. However, the naming is misleading. The Pimax 8K (pre-release test) has two displays in 4K resolution, not one in 8K. Moreover, the resolution is upscaled to 4K. The predecessor model, the Pimax 4K, sold around 30,000 units within half a year.

VR arcade “The Void” opens in Utah, Windows Mixed Reality launches and Sony improves PSVR

Meanwhile, the first official "The Void" arcade is opening in Utah. The VR arcade experience combines virtual and actual reality. To do so, designers recreate the virtual playing field in a physical space equipped with real objects such as walls and torches.

If a Void player reaches for a virtual torch, the analog counterpart is in the same place in reality - including heat emitters. This makes the VR world much more believable with haptic elements. Even VR freaks can't recreate the Void experience within their own four walls. What's more, The Void cooperates exclusively with strong entertainment brands like Ghostbusters and Avengers.

Microsoft releases the long-awaited Windows 10 "Fall Creators Update." It includes the "Windows Mixed Reality" portal, originally announced under the name "Windows Holographic". The first Windows Mixed Reality headsets release in the fall and make a solid start with plenty of upside potential.

Each of the VR headset brings a resolution of 1,440 x 1,440 pixels to two 2.89-inch LC displays with 90 - 100 degree field of view. Sony launches a revised model of its PlayStationVR. Model number CUH-ZVR2 hits stores with a new headset mount directly on the glasses, a revised external processor unit including simplified cabling, and minor design tweaks.

2018: The birth of a VR superhit

With Beat Saber (review), one of the most important VR games of the coming years appears in 2018 and immediately becomes a phenomenon. Less than a month after its release, the game has already sold more than 100,000 copies and generated sales of $2M.

The concept: armed with two laser swords, players slice up approaching cubes to the rhythm of electronic music. The developers hit the bull's eye with this. Beat Saber integrates VR movement into the gaming experience more simply and elegantly than any other game before it. It only works in VR; there is no comparable experience on the monitor.

The VR Institute of Health and Exercise state that a Beat Saber workout is similar to a game of tennis - with an energy expenditure of eight to ten calories per minute. Players around the world share their weight loss successes via social media. Beat Saber appears on all major VR platforms and is becoming a big seller on each one.

The beginning of standalone headsets

With the HTC Vive Focus, the Taiwanese are launching the first self-sufficient VR headset at the end of the year and want to score points primarily in the B2B sector and in education. The required hardware fits entirely in the casing of self-sufficient VR headsets. External devices such as PCs, smartphones, or a console are superfluous. The simple operation and mobility should make the devices more interesting for VR novices.

Before the Vive Focus, the Oculus Go releases in Germany in June. It sets standards in terms of mobility, ease of use, and price-performance ratio. However, since the VR headset does not recognize movement in the depth of space and only supports 3DOF (Degrees of Freedom), its use consists entirely of 360-degree videos or basic games.

This is exactly where Lenovo starts its cooperation with Google. The Lenovo Mirage Solo is a standalone VR headset with precise 6DOF tracking. In contrast to the Oculus Go, walking in virtual reality is possible here - but only that. The included controller remote only allows 2D hand movements in three-dimensional space.

This inconsistent hybrid approach also leads to a VR experience with very limited immersion in the HTC Vive Focus. The virtual hands and the corresponding interactions are missing, and they are at least half of the VR fun - especially in VR games.

Upgrade for HTC Vive and announcement of Oculus Quest leads to next rift at Facebook

Facebook CEO Mark Zuckerberg announces the standalone Oculus Quest VR headset at Oculus Connect 5. This is supposed to come close to the performance of an Oculus Rift in the VR core experience and make VR even more accessible.

According to Oculus technical director John Carmack, the Oculus Quest should deliver decent graphics despite mobile hardware - if developers focus on the essentials. Carmack cites the Nintendo Switch as a possible competitor.

Facebook's focus on accessibility instead of maximum performance is not what Oculus co-founder Brendan Iribe has in mind. He reportedly fundamentally disagreed with Oculus' direction and wanted the company moving further in the direction of high-end VR. Facebook's push with the Quest, probably a heart project of Facebook CEO Mark Zuckerberg, and an alleged cancellation of a high-end VR headset Oculus Rift 2 reportedly led to the final rift.

Former Oculus CEO Brendan Iribe is the second Oculus founder to leave Facebook after Palmer Luckey. Meanwhile, HTC remains true to its PC-based high-end VR headset and is giving it an upgrade. With the HTC Vive Pro, HTC answers one of the most ardent wishes of VR enthusiasts - more pixels. The new 3.5-inch OLED displays deliver a resolution of 1,440 x 1,600 pixels per eye at 90 Hz.

2019: Oculus Quest, Valve Index, and the announcement of a gamechanger in the VR gaming market

HTC releases Vive Cosmos and Vive Focus Plus.

HTC announces its new Vive Cosmos VR headset at CES 2019. There is talk of the sharpest display to date and minimal fly screen effect. At the same time, the VR headset is to become simpler and more mobile. Integrated cameras take care of spatial orientation, and there is no need for an external tracking system.

In June, HTC announces exact specifications. 1,440 x 1,700 pixels per eye, two LC displays with RGB subpixel matrix for a reduced fly screen effect and integrated adjustable interpupillary distance.

However, the VR headset is not well received. After the release in October, testers mainly criticize the imprecise tracking and the clunky controllers. Two or three hours drain a total of four AA batteries. The tracking also does not work in poor lighting conditions. HTC responds to the criticism and vows to improve. The Cosmos has not yet reached the success of the Vive generation.

The HTC Vive Focus Plus is an upgrade to HTC's standalone VR headset. Besides eye tracking, the Vive Focus Plus also gets new, exclusive controllers. They support movements in all directions and thus fix a major point of criticism of the predecessor: the lack of virtual hands.

Sony tops the sales charts and rumors about a successor

Sony files patents, sparking rumors of a PSVR 2. New hand controller design, inside-out tracking and a new version of the PlayStation tracking camera indicate the direction Sony could take with a possible PlayStation VR 2 (info).

The announcement of the new PlayStation 5 gaming console promises support for, it's clear that PSVR. PlayStation VR 2 does not launch with the new console. Back in March, Sony announced that the PlayStation VR has sold over 5 million units to date, making it the most successful VR headset on the market.

Facebook: Oculus Rift S, Oculus Quest and the farewell of a developer legend

Facebook's next PC VR headset releases in the spring 2019. The Oculus Rift S is more of an upgrade than the successor to the first Rift. A new LC display with 1,280 x 1,440 pixels per eye, revised controllers, and a tracking system integrated into the VR headset are the biggest differences.

With the Oculus Quest, Facebook launches a second VR headset in 2019. It works completely standalone, but can also be converted into a PC VR headset via Oculus Link (info). Compared to the Rift S, the Oculus Quest has higher-resolution display and offers more flexibility as well as a wireless VR experience in standalone mode.

The Quest's two OLED screens bring 1,600 x 1,440 pixels to each eye. The headset supports two Oculus Touch controllers, though Quest hand tracking (info) is out of its initial experimental status and is an official part of the Oculus Quest's feature list.

The success of the Quest exceeds expectations and Facebook wants to invest further in VR. However, Oculus chief technology officer John Carmack is no longer sufficiently excited by these successes. Only days after receiving a VR award for his life's work, he resigns as Oculus Chief Technology Officer and dedicates himself to the development of a super AI from now on. He plans to continue to assist Oculus in an advisory capacity.

Meanwhile, Oculus founder Palmer Luckey expresses frustratration with the latest developments at Oculus. The Rift S is great, but not usable for him and another 30 percent of potential customers.

His eye relief is too great, as the Oculus optimized Rift S for an eye relief of about 64 millimeters. In any case, he said, the current generation of VR headset is "still too expensive as a gift" - the technology needs to get much better before it meets the sci-fi expectations of many people.

Valve launches its VR headset, the Valve Index, and announces Half-Life in VR

Valve decides to go it alone and develop its own VR headset. This gradually makes the cooperation between HTC and Valve obsolete.

The Valve Index opens for pre-orders on May 1, 2019. It sells out just one day later. High resolution, crisp colors, variable frame rates, innovative controllers, excellent tracking and high wearing comfort make the Valve Index (review) one of the best PC VR headset on the market.

However, excellent VR headset also needs excellent content. In November, Valve officially announces Half Life: Alyx, continuing the immensely popular Half Life franchise exclusively in virtual reality. Half a year later, Half Life: Alyx releases and becomes (arguably) the best VR game to date.

VR in the 2020s: Facebook vs. Germany and VR in the Year of the Pandemic

2020: Virtual Reality in times of the pandemic

The year 2020 primarily tells the story of a pandemic that affects all areas of people's lives. The deadly virus Sars-CoV-2 spreads worldwide and cripples the economy in many countries. People quarantine and working from home becomes the norm.

The VR industry aims to meet the growing demand for video calling and online meetings with telepresence apps. More than 150 apps for XR collaboration appear in an online database at the end of the year. Apps like Spatial or Immersed can't really catch on compared to Microsoft Teams, Zoom, or Skype. Industry observers see the technology as too cumbersome for VR meetings.

Social VR apps benefit much more from the pandemic restrictions on public life. According to Facebook, Oculus Quest usage numbers have increased during the pandemic. Fitness and social applications in particular, explode in popularity during the quarantine.

Half-Life: Alyx provides interim VR high

Valve's Half-Life: Alyx does immensely well with critics and fans, scoring 93 out of a possible 100 on Metacritic. Critics celebrate the game as a "gaming milestone" and "the best VR game". Valve manages to create a believable virtual world that has an unprecedented level of detail with many interaction possibilities.

The game's success provides an interim VR high, but does not decisively advance the industry. As measured by SteamVR statistics, PC VR userbase and activity has stagnated since the game's release. By year-end 2020, about two percent of all SteamVR users own VR headsets.

Facebook stamps out Oculus Rift (S) - the future belongs to Quest

Too little for Facebook: The blue corporation announces the end of the PC VR headset Oculus Rift (S). In addition to the arguably saturated PC VR market, the triumph of the standalone VR headset Oculus Quest is likely a decisive factor in the demise of the Oculus Rift (S).

The Quest generated just under $100M in  in-app revenue within the first twelve months of launch. That's more than the Rift (S) and Oculus Go ecosystems combined generated in three and a half years.

Facebook continues to invest in VR and releases Oculus Quest 2

Driven by the success of its autonomous VR headset, Facebook continues to invest aggressively in the VR market. In addition to well-known VR game companies such as Sanzaru Games and Ready at Dawn, money is primarily flowing into research. Somewhat surprisingly, Facebook releases the Oculus Quest 2 (review) in October 2020 - just under a year after the release of its predecessor.

Facebook improves the Quest 2 on all fronts: LC display with 1,832 by 1,920 pixels per eye, refresh rates of up to 120 hertz, better CPU, more memory, and a low price. Despite many technical improvements, the Oculus Quest 2 costs $100 less than the Quest 1.

CEO Mark Zuckerberg wants to reach more than ten million active users in the next few years. However, data protectionists accuse him of wanting to use inexpensive hardware to make it easier to obtain valuable VR data for AI research. Anyone who buys an Oculus Quest 2 can only operate the VR headset with an active Facebook account.

Facebook vs. German Federal Cartel Office: the Oculus Quest 2 cannot release in Germany

Facebook's opaque data policy is taking the social media and tech company to several German courts. Since February 2019, Facebook has been in a legal dispute with the German Federal Cartel Office and has to justify its excessive hunger for data. In the Oculus case, the connection of Oculus and Facebook accounts in particular, and a thus illegally imposed coupling of two different services, is causing trouble.

Facebook decides on its own initiative not to release the Oculus Quest 2 in Germany and to stop selling Oculus headsets there. This seems to be Facebook's way of escaping potential legal problems without having to incur financial losses in the distribution of the devices. German users can still obtain Oculus VR headsets via other EU countries and operate them without any problems.

End of the Oculus Go and new high-end VR headset

According to Facebook, the Quest 2 sells better than the original. The Facebook account requirement hardly seems to deter customers. The success of the Quest 2 also means the end for the Oculus Go. The 3DOF headset clearly passed its zenith as several other headsets surpassed it technologically. In addition to Facebook's self-powered Quest 2, many more VR headsets hit the market in 2020.

The StarVR One appears in small numbers after initial problems and threatens solvency of the developers. The high-end VR headset has an integrated eye-tracking module, an extra-wide field of view with up to 210 degrees horizontally and 130 degrees vertically, and two AMOLED RGB displays. These resolve with 1,830 by 1,464 pixels per eye and three subpixels per pixel, at a refresh rate of 90 Hz.

Finnish manufacturer Varjo launches two high-performance premium VR headsets, the VR-3 and XR-3, aimed at small and medium-sized businesses. The third-generation device's field of view increases from 87 degrees to 115 degrees horizontally. There is an augmented reality mode, hand tracking from Ultraleap, and optimized Varjo-type dual display.

HP Reverb G2 serves VR simulation fans and an old acquaintance returns

In addition to the high-end devices from Varjo and StarVR, a consumer VR headset, the HP Reverb G2 (review), comes onto the market. The G2 delivers one of the sharpest VR images ever with a resolution of 2,160 by 2,160 pixels per eye. However, the VR headset shows weaknesses in controller tracking and is therefore mainly suitable for simulation fans.

A former VR great also returns at the end of the year: HTC wants to continue old successes in 2021 and announces several new VR headsets. There is talk of "groundbreaking devices" that will "excite people in the same way as the first Vive." However, the Vive Pro 2 (review) and the truly novel Vive Flow unfortunately only score limited points in tests.

2021: Sony announces the PlayStation VR 2 and the Metaverse hype cycle begins

After many rumors, it becomes official in early 2021: Sony will release a PlayStation VR 2. In February, Playstation CEO Jim Ryan announces a "completely new VR format" for the Playstation 5. Sony plans to ship Playstation VR 2 developer kits soon with a full release expected in 2023.

Just a few weeks later, Sony presents the new VR controllers for the PSVR 2 and thus buries the unloved PS Move controllers of the predecessor. The PSVR 2 controllers appear in a completely new, ergonomic design, which, according to Sony, results from tests with VR users with different-sized hands. In addition, the DualSense technology of the Playstation 5 controller supposedly increases the immersion significantly.

Rumors about Apple’s VR headset grow stronger

There are no official statements from Cupertino so far, but the rumors that Apple is working on its own VR headset are becoming more and more solid. A Bloomberg report states that Apple will announce a mixed reality headset "in the next few months". Apple's annual developer conference falls within that time frame but fails to deliver on the rumors.

But Apple doesn't talk about its own VR/AR headset at WWDC 2021 either. Instead, there are more rumors. Apple expert Ming-Chi Kuo claims Apple's XR headset is on the horizon. Rumors claim it has up to 15 cameras installed, to offer an AR mode via video review in addition to the VR mode, and offer eye-tracking and foveated rendering.

Facebook is now Meta and is gearing up for the Metaverse

While Apple fans won't hear any news about the company's VR/AR future in 2021, Mark Zuckerberg has several big news items at Facebook Connect 2021. Facebook is now Meta and has a new high-end VR headset in the bag.

The new name reflects the company's identity and future focus on the Metaverse. Zuckerberg said he hopes that, over time, the public perceives the group as a metaverse company.

"We chose Meta because it can mean 'above' and describes our commitment to developing social technologies that are above what is possible with digital networks today," Zuckerberg said.

Meta's focus is thus clearly on the Metaverse, virtual reality, augmented reality, and Oculus, the VR headset maker Facebook acquired in 2014. Still, the Oculus brand name is gradually disappearing as part of the rebranding. The Oculus Quest 2 became Meta Quest 2, the Oculus logo gradually gave way to Meta's curved eight, and the Oculus mobile app became the Meta Quest app.

Project Cambria: These are Meta’s new VR headset

The second big attraction at Facebook Connect 2021 was the Project Cambria VR headset. It is not a Meta Quest 3. Nor is a Quest 2 replacement. Cambria, the company said, is a high-end device that will be more expensive but "packed with the latest technology."

Meta announced an AR mode for the Cambria that, unlike the Quest 2, should provide an accurate image of the environment. Eye and face tracking are said to provide better social presence in virtual reality. In addition, there is a new optical system based on pancake lenses. Project Cambria is scheduled for release in 2022.

Cover image: By Dr. Waldern/Virtuality Group - Dr. Jonathan D. Waldern, CC BY-SA 4.0, link.

Note: Links to online stores in articles can be so-called affiliate links. If you buy through this link, MIXED receives a commission from the provider. For you the price does not change.