Inventing the familiar: "Damocles-Headsight-View"
Using cameras to display reality in a headset is a long-established technology. Is a camera-monitor system as AR glasses still a timely idea?
Author: Armin Grasnick
AR glasses are reality augmentation devices that enrich real environments with virtual elements. Since Ivan Sutherland's "Sword of Damocles" (Sutherland, 1968), such systems have been based on the principle of overlaying the real with the virtual.
A computer-generated object is visually introduced into the viewer's field of view and blended into the environment. However, a strong sense of telepresence in distant or virtual worlds is only achieved when the visual impression follows the head movement and obscures the local view. It is obvious to combine the head position with the control of stereoscopic cameras in order to achieve a visual impression as realistic as possible.
1960s Headsight Television
The innovative U.S. company Philco had been making radios and televisions since the 1950s. In the early 1960s, two Philco researchers integrated one of these portable televisions into a video headset.
Movement of the head controlled a remote camera, whose image was transmitted to the tube and displayed to the viewer. The following picture shows Philco researcher Charles Comeau wearing such a headset.
Philco researcher Charles Comeau wears a VR helmet. | Image: Charles Comeau, 1961, from (Comeau & Bryan, 1961, p. 87)
The "Headsight" was a less elegant helmet headset, but still a groundbreaking, functional and, above all, inspiring prototype.
1990s NASA View Headset
Such systems have been experimented with since the late 1980s, for example in the Virtual Environment Workstation Project (VIEW) (Bolas & Fisher, 1990). NASA's View headset from 1992 is already quite close to the design of today's VR headsets.
This system could not only remotely control a stereoscopic camera, but also display either the real 3D scene or a virtual environment. The picture shows the full version with 3D sound and gloves for real interaction with the telepresent world.
NASA's View headset was released in 1992, but it already looks a lot like today's VR goggles. | Image: Wade Sisler, 1992, NASA images.nasa.gov/details/ARC-1992-AC89-0437-6
One more thing
It has been a long road from what has been presented as a prototype for decades to a consumer-ready device. Only with the introduction of the Oculus VR glasses in 2012 and their focus on VR content did they enter the consumer market. At the same time, there was a greater division of the technology into Augmented and Virtual Reality, with their respective headsets and corresponding benefits and problems.
An obvious disadvantage of VR headsets is the lack of a view of the real world, which sometimes leads to an unwanted and potentially dangerous isolation from the outside world when moving. AR glasses provide a direct view of the real world, but cannot display opaque objects. A real object can only be obscured by light.
To improve the ghostly AR displays and to enable the VR view of the environment, cameras are used again. It is obvious and immediately apparent that the insertion of the external camera view leads to the apparent see-through of the display and can only be done via the video signal. Consequently, the term "video see-through" is as superficial and unnecessary as the term "optical see-through". It's like calling walking barefoot "haptical-feel-through" and wearing a shoe "protected-feel-through" just to avoid using uncool terms like "barefoot" or "shoe".
Practically speaking, all video passthrough technologies are systems that capture images with cameras and display them on monitors. This is the same regardless of the manufacturer, whether it is Meta, HTC, Varjo, or Apple. We might be tempted to call new VR/AR headsets with external cameras "Damocles-Headsight-View" (DHV) or "Comeau-Sutherland-Fisher" (CSF) to honor the decades of groundwork, but for simplicity's sake we'll stick with the unambiguous term "Camera-Monitor-System" (CMS) for the "video-passthrough" part.
VR headsets as a Camera-Monitor-System
But why are VR goggles with the option to switch the external view a good idea? The classic advantage of telepresence is hardly used currently because the controllable stereo cameras of the remote station are clearly underrepresented compared to VR headsets. In professional applications, however, this option may become relevant again for reasons of employee safety and to avoid real visits to unpleasant environments.
A much broader advantage is the high-quality possibility of blending the outside world without ghosting. This leads to effective presence of virtual parts and can increase the feeling of mixing real and virtual worlds.
A CMS of VR headsets consists of the critical components lens, camera sensor, computer, display, and eyepiece. Each component has an impact on the representation of the real scene and introduces latencies into the system.
It would be desirable to have a standard of comparison for the image quality of a CMS that is independent of a manufacturer's definition. This is where classical optics comes in, which knows several imaging errors (aberrations). These can be objectively determined and used to compare different systems. This can be done quite simply by taking different test images directly through the eyepieces of the eyeglasses using a high-resolution camera.
CMS Transfer Function
In principle, the change in an input signal as it passes through the system can be determined by a transfer function that is both time and location dependent. The transfer function would then be the black box function that leads to the measured output signal for a known input signal.
Optical transmission testing works very well with appropriate test patterns, such as those used in camera testing. It is not necessary to develop your own tests for this purpose; appropriate standards for other test cases can be found here. Of particular interest is the European standard ISO 16505, which was originally developed for the replacement of exterior mirrors on road vehicles. Passing a test according to this standard is necessary in order to obtain road approval for a camera-monitor system in a vehicle. This should then be sufficient to qualify a less dangerous device such as AR/VR glasses for use.
Will the sword fall?
Using cameras in XR headsets brings real benefits in terms of display quality. This is especially true when VR headsets are used for AR viewing. However, it is possible that this theoretical quality gain is reduced by imaging errors during recording and playback.
Apple's Vision Pro can be called a camera-monitor system, and will have to be judged as such in the future. The sword of Damocles of a possible rejection by the market is also hanging over Apple. An objective measurement will not change this, but determining the transfer function can lead to greater comparability and help users choose a headset.
The true quality can only be evaluated on a real device. However, until the Vision Pro has been measured, we will not remove the sword. Reality will show if the "Damocles Headsight View" really deserves the name Vision Pro.
Armin Grasnick is Professor of Augmented & Virtual Reality at IU International University. He has been working on VR with and without glasses since the 1990s.
Note: Links to online stores in articles can be so-called affiliate links. If you buy through this link, MIXED receives a commission from the provider. For you the price does not change.
