Eye tracking via sonar: Researchers hope to make VR headsets more efficient
Sonar instead of cameras: Researchers develop energy-efficient and private eye-tracking technology for VR headsets — but not without drawbacks.
Researchers at Cornell University in New York have developed prototypes of a sonar-like technology that could replace cameras for tracking eye movements. It uses tiny speakers that emit sound for each eye at frequencies above 18 kHz. These are beyond the hearing range of most people.
The sound is directed at the face, reflected, and picked up by four microphones on each side of the headset. An algorithm the researchers call GazeTrak then interprets these sound waves to determine the direction of the wearer's gaze.
Cheaper, lighter, but less accurate than cameras
According to the research team, sonar technology should offer several advantages: It reduces power consumption compared to camera-based systems and offers users more privacy because cameras are not constantly recording. It could also reduce the manufacturing costs and weight of VR headsets.
In tests with 20 participants, sonar-based eye tracking showed an accuracy of up to 3.6 degrees. This is not as accurate as current high-end devices such as the Apple Vision Pro. However, the researchers believe that this performance should be sufficient for most Virtual Reality applications.
So far, the system has one major drawback: Because the shape of the eyeball differs from person to person, the AI model used by GazeTrak has to be trained separately for each user. To commercialize the eye-tracking sonar, enough data would have to be collected to create a universal model.
Precise menu navigation, foveated rendering: Eye tracking offers numerous advantages
Eye tracking is a key technology for Virtual Reality, allowing you to navigate menus by looking at specific points or make eye contact with other avatars in virtual environments. Apple Vision Pro is currently demonstrating how precise eye tracking can improve the user experience.
Eye tracking also enables rendering that takes the user's focus into account by providing a detailed representation of the area being viewed and a less detailed representation in the periphery (foveated rendering), for example on the Playstation VR 2. The technology also enables innovative control methods, such as the VR game Before Your Eyes, which you control solely with your eyes.
Eye tracking will eventually become standard
Despite these advantages, market leader Meta has decided not to use the technology in its current Quest 3 mobile VR headset, citing manufacturing costs, weight, and the computing power required. So far, Meta has only used eye tracking in the Meta Quest Pro.
But Metas CTO Andrew Bosworth believes that eye tracking will one day become standard:
"We have really been testing it for a long time as a 2D navigation UI paradigm in conjunction with hands. And so we've known this for a long time. We've been pursuing this for a long time. It is just a trade-off on cost and weight and obviously, extra sensors, against the product. So we're really focusing on continuing to be able to deliver it at high degrees of precision in more efficient architectures, both computationally and price standpoint, so we can make it into every single headset. I can't tell you when it's going to happen. Every single headset is kind of its own recipe, but at some point yes, it will be part of the base package. I'm not sure exactly when, though."
Note: Links to online stores in articles can be so-called affiliate links. If you buy through this link, MIXED receives a commission from the provider. For you the price does not change.
