Headset eyes could solve VR's social problem - or scare you to death
"Reverse passthrough" is getting more compact - and more creepy. A Japanese tech lab shows the bloodshot eyes of headset wearers with flat round screens on the outside of the headset.
How do we move from a closed-off virtual reality experience to social interaction between VR users and others in physical space? That's the question many VR headset engineers are asking. Gestures, facial expressions, and especially subtle eye movements are, after all, an important part of human communication.
Even modern mixed reality headsets with passthrough AR, like the Quest Pro, blend the real and virtual worlds only for the user: outsiders see at best a 2D image of the XR application and barely the face of the person using it.
Quest 2 Frankenstein's Eyes Edition
A year and a half ago, Meta researcher Nathan Matsuda already demonstrated an initial solution for making eye contact with eye tracking. Meta's prototype for so-called "reverse passthrough" has two screens on the outside of the case that mimic the user's eye movements in real time. This is intended to enable eye contact during VR use.
This feature is called reverse passthrough because a video image is streamed from the headset to a display outside the headset. With conventional passthrough, an image of the environment is streamed into the headset from the outside.
Metas current reverse passthrough prototype serves primarily as a proof of concept - or sci-fi prop in a Terry Gilliam movie. | Bild: FRL Research
However, an early feasibility study by Meta had the elegance of a wired Borg drone from Star Trek. Another model with only one screen was reminiscent of a tube TV that you strap in front of your face.
Meta's light field reverse passthrough prototype. | Bild: Meta
The R&D Lab at Nippon Television (NTV) proves that things can be even more sinister. The research and development team of the traditional Japanese private broadcaster recently presented a Meta Quest 2 with two circular displays glued to the front.
They show two deadpan eyeballs covered with red veins and without eyelids. A connection, including a circuit board, leads from the display eyes to an external computer that feeds the eye animations.
円形ディスプレイ使って、Quest2に目をつけてみました！！良い感じ。次はアイトラッキングして視線と連動させたい。目を合わすのが苦手な人もこれで解決？#quest2 #xr pic.twitter.com/OZ98swwOBI
— 日テレR&Dラボ🔜CES2023 (@ntv_rd) December 21, 2022
Meta's solution looks almost realistic in comparison.
Video: Meta
The next step is to synchronize digital and real eyes with eye tracking
The wide-open, glowing eyes on the front do not yet imitate the real-time movements of the eye in the headset: Eye-tracking support with gaze tracking is to follow in the next step, according to NTV's R&D Lab. HTC Vive promptly offered cooperation on Twitter: "We would like to see it with the eye tracking of the Vive Focus 3," writes a Japanese Vive account for Developer Relations.
Note: Links to online stores in articles can be so-called affiliate links. If you buy through this link, MIXED receives a commission from the provider. For you the price does not change.
