This is as close as you can get to petting a cat on Meta Quest 3
A VR developer shows in a video how passthrough, hand tracking, and simulated physics can create the illusion of petting a cat.
Dennys Kuhnert posted a video of his augmented reality cat on Twitter last week. The virtual pet follows his hands with its eyes and meows and purrs as he strokes it.
The video shows the view through a Meta Quest 3. The color passthrough of the device shows the physical environment, making the experience more realistic.
Note how the virtual hands physically adapt to the cat's body. Kuhnert is a hand tracking virtuoso and the creator of Hand Physics Lab, the first VR hit game for this new controller-less way of playing.
Anyone who has tried Hand Physics Lab before knows that a simulated physical resistance can trick the brain into thinking it is dealing with inert objects. The same could be true for the cat, which is really just air.
First mixed reality experiments
Attentive observers may wonder why Kuhnert is projecting skeletal hands onto his real hands.
The reason is that Meta Quest 3 does not yet support real-time dynamic occlusion, which means that digital objects like the virtual cat are always rendered in front of moving objects like your hands. This destroys the illusion of having a physical object in front of you. If you look closely, you can see that Kuhnert's real hands disappear behind the cat. The digital skeletal hands from Physics Lab counteract this effect.
Meet my augmented reality cat ? pic.twitter.com/PlfCGgtkzV
- Dennys Kuhnert (@DennysKuhnert) October 19, 2023
Kuhnert writes me that the virtual cat is not new and was implemented in Hand Physics Lab years ago (spoiler alert: the cat appears in the very last level).
The developer says the video shows one of his latest mixed-reality experiments. And he can't say where it's going yet. "I'm primarily experimenting and prototyping with AR since a few months," Kuhnert writes to me. "Real-time hand occlusion for AR and depth sensing are the next thing I will experiment with."
Mixed reality Tamagotchis are coming soon
Virtual pets could be an interesting new use case for mixed reality headsets. A group of Oculus veterans have even started a studio dedicated to creating digital companions.
"Modern game AI is capable of engagement and responsiveness far beyond what Tamagotchi offered, and modern game engines can deliver performances far more nuanced than Sony’s robot dog — but until now, a truly believable pet sim was missing something," the studio writes. That missing element is virtual and mixed reality. The technology, the startup hopes, will make the virtual companions so realistic that users will develop an emotional attachment to them.
We'll have to wait and see. After all, you have to wear a headset to see the pets. But they should be enough for a fun pastime, and not a hindrance if you are allergic to cat and dog dander.
Note: Links to online stores in articles can be so-called affiliate links. If you buy through this link, MIXED receives a commission from the provider. For you the price does not change.
