Will this AI system solve Meta's full-body avatar problem?
A study by Meta AI shows a new avatar model that realizes smooth human full-body movements in virtual reality – without additional sensors.
The inside-out tracking of mobile VR headsets like the Meta Quest 2 or Quest Pro mainly captures the user's head and hands. Because the tracking cameras are integrated into the headset's housing, the rest of the body is largely hidden from motion detection.
In a study, Meta AI presents a new avatar system that synthesizes fluid movements of the entire body from this sparse data and thus significantly outperforms previous avatar animations.
AI generates fluid movements in real time
“Avatars Grow Legs”, or “AGroL” in short, is a diffusion model specifically designed to track whole body movements with only a few upper body signals. It is based on a Multi-Layer Perception (MLP) architecture and a novel conditioning scheme for motion data.
According to the researchers, it should be able to predict precise and uniform full-body movements, which could solve the problem of VR full-body tracking. The lower body, which is difficult for conventional tracking, should not be a problem. Only when touching the ground, artifacts occasionally occur.
Since AGroL can be executed in real time, it should also be suitable for online applications. This means that the less realistic meta-avatars without legs in Horizon Worlds and other social VR applications could soon be a thing of the past.
AGroL: Smoother than AvatarPoser
The researchers demonstrate the effectiveness of the model using a motion capture dataset from AMASS. Compared to other avatar systems such as AvatarPoser, AGroL has significantly fewer rotation, position, and velocity errors.
The generated movements are therefore more precise and fluid. Above all, the jittering of the virtual arms and legs known from previous avatars is said to occur much less frequently with AGroL. In the YouTube video above, you can see a comparison between AGroL and AvatarPoser starting at minute 1:44.
Note: Links to online stores in articles can be so-called affiliate links. If you buy through this link, MIXED receives a commission from the provider. For you the price does not change.
