Canon puts your face in VR calls with Quest app

Canon puts your face in VR calls with Quest app

Canon's "Kokomo" brings more personality to VR meetings. Instead of VR headsets, the avatars "wear" scanned faces.

Interact with a friend as if they were standing right in front of you. This is the promise of Canon's Kokomo VR app. Video conversations take place in an immersive virtual reality environment and offer a special feature for more personality.

Although the person you're talking to is actually wearing a VR headset, an animated image of their face appears in the virtual world. In February 2023, the software entered a closed early access phase for Meta Quest, iOS, and Android.

VR calls with the Quest 2

Getting started is easy. All you need are a VR headset and a supported smartphone. The companion app on the cell phone captures the camera image of the conversation partner and displays it against a virtual background, similar to a filter in platforms like Zoom.

A realistic avatar in Kokomo against a beach background.

The body captured by the phone's camera is given a kind of photo wallpaper of the face - roughly where the VR headset actually sits. | Image: Canon

The smartphone and app scan your face to create a 3D model and superimposes this model over where the headset sits on your face making it appear as is you are not wearing a headset at all during Kokomo calls.

Before starting a call, position the smartphone at waist height so that your entire body is in the camera frame. Then, start the call. During the call, navigate a VR menu using hand-tracking on Quest. That way, you're not holding controllers in your call after all that work to hide your headset.

Techcrunch contributors already tested the software extensively and praise the scaling options. That means that you can adjust the proportions to match your virtual counterpart without looking creepy.

"If everything goes to plan, it’s telepresence at its finest, without the vast expenditure of typical telepresence systems," said Techcrunch editor Haje Jan Kamps.

logo

In practice, however, this is not always currently the case. In some initial test runs, the jerky image dropped to three to four frames per second. Further, the full-body avatar appears two-dimensional, sometimes resembling an animated cardboard cutout.

"Face mask" in virtual reality

Also, the "mask" of the face superimposed on the image kept disappearing and reappearing, which regularly broke immersion. If Canon gets all of these problems under control during Early Access, the concept is very promising.

Backgrounds include living rooms or beach settings like Malibu and Hawaii. You can also share pictures or videos with your conversation partner.

Kokomo is available for free on Quest, Quest 2, and Quest Pro in the App Lab. The companion app is already available for iOS and Android.

Buy Quest 2, Quest Pro & Prescription Lenses
Quest 2
Quest Accessories
Quest Pro
VR Optician
Sources: Canon, Techcrunch