New eye tracking method could dramatically improve VR headset accuracy
University of Arizona researchers have developed an eye-tracking technology that captures over 40,000 surface points of the eye — a massive leap forward for VR precision.
Researchers at the University of Arizona have unveiled a novel eye-tracking technique based on deflectometry. While traditional infrared emitter methods only capture about twelve points on the eye's surface, this new technology collects data from more than 40,000 surface points — all from a single camera shot.
Deflectometry is a 3D imaging technique typically used for measuring reflective surfaces with extreme precision. The scientists use a screen displaying special light patterns to illuminate the eye. Cameras capture and analyze how these patterns reflect off the eye's surface. Any of the over a million pixels can serve as individual point light sources.
By analyzing how the displayed patterns deform when reflected off the eye's surface, the system creates a detailed 3D representation of the eye, capturing information from both the cornea and the white area around the pupil.
In human trials, the system achieved tracking accuracies between 0.46 and 0.97 degrees. Tests with an artificial eye model showed even better results, with errors as low as 0.1 degrees. The university has already filed a patent for this technology and is planning commercial implementation. The team is now working to integrate additional 3D reconstruction methods and artificial intelligence to further advance the technique.
Lead researcher Florian Willomitzer emphasizes the advantages: “With our deflectometry-based method, we can use the information from more than 40,000 surface points, theoretically even millions, all extracted from only one single, instantaneous camera image.” Postdoctoral researcher Jiazhang Wang added: “More data points provide more information that can be potentially used to significantly increase the accuracy of the gaze direction estimation. This is critical, for instance, to enable next-generation applications in virtual reality.”
Eye tracking is hugely important for XR
Eye-tracking is crucial for advancing VR and AR headsets. It enables not only realistic avatars with eye contact, but also intuitive controls like those used in Apple Vision Pro.
The technology is particularly important for Dynamic Foveated Rendering, which detects which part of the field of view the eye is focusing on and allocates computing power accordingly. This can significantly improve graphics quality, as demonstrated in Playstation VR 2.
However, integrating eye-tracking into standalone VR headsets like Meta Quest remains challenging. Meta's CTO Andrew Bosworth explains: “To effectively use Dynamic Foveated Rendering, you need extremely precise eye-tracking. Interestingly, achieving such accurate eye tracking currently requires more computing power than you save through foveated rendering.”
The new deflectometry method could represent a breakthrough here. The researchers have demonstrated that both accuracy and precision increase with the number of data points collected. If this method can be efficiently implemented, it could pave the way for more effective eye-tracking that's economically viable even in mobile VR headsets.
What do you think of the new method? Join the conversation on Facebook, Bluesky or X or share your opinion in the comments below.
For feedback, topic suggestions, or other ideas, please email us at hello@mixed-news.com.
Note: Links to online stores in articles can be so-called affiliate links. If you buy through this link, MIXED receives a commission from the provider. For you the price does not change.
