Quest Pro turns your eyes into a keyboard
Meta Quest Pro opens up new possibilities. A developer shows how you can write in virtual reality using eye tracking.
In a video published on YouTube and other channels, iOS developer and XR enthusiast Fabio Dela Antonio demonstrates an eye-tracking experiment. Antonio's eyes dart over each key on a virtual keyboard. Little by little, the sentence "Hello World" emerges. Antonio selects the characters through eye tracking on a Quest Pro (review).
The input occurs as soon as the gaze rests on a virtual key for half a second. The developer does not need more than his eyes. In the video, Antonio holds his hands in front of him to prove that he is not secretly typing on a keyboard after all.
"Blipping" is not a keyboard replacement
As interesting as the idea of gaze-controlled typing or "blipping" may seem, it is unlikely to replace physical typing. Typing simply takes too long for that. For example, the 12 seconds it took for Antonio to type the words "Hello World".
It would be a bit faster if a controller key could be used for confirmation. However, preserves the second problem of this approach: the necessity of looking at a keyboard while typing. This is no longer necessary with physical typing, which also enables fast typing in the case of the ten-finger system.
Such a feature would still be useful, though. For example, as an alternative for the more cumbersome typing via VR controller and virtual laser pointer, which is still the standard input method in Meta Quest 2 (review) and Pro.
A big win would be the gaze-controlled typing for people who can't use their hands for typing due to a disability. This is exactly the group Antonio had in mind. He hopes that Meta will bring such an accessibility feature to Meta Quest Pro.
In search of the XR keyboard
Efficient typing in VR and AR is an unsolved problem. In addition to the laser pointer approach mentioned above, Meta Quest 2 & Pro has the option to bring the physical keyboard into VR. This only partially solves the problem. After all, we want to do without a real keyboard one day. For example, with the help of a wristband that turns any surface into a keyboard.
Meta pursues an AI-supported finger-tracking keyboard for this purpose and also demonstrated a system called "Pinchtype", in which inputs come from pinch gestures. This approach is innovative but takes practice.
Speaking of artificial intelligence, the Slide-To-Type system known from smartphones, where you drag your fingers across the keyboard and an AI model guesses the intended word, could simplify and speed up Antonio's gaze-controlled typing. In this case, it might be enough to let your eyes wander over the sequence of letters.
You can download his prototype from Github and install it on Meta Quest Pro via Sidequest (Guide).
Note: Links to online stores in articles can be so-called affiliate links. If you buy through this link, MIXED receives a commission from the provider. For you the price does not change.
