Meta and TU Munich turn smartphone pics into lifelike 3D avatars in minutes
Researchers have developed a new method to create an animatable 3D avatar from just four smartphone photos.
The system, called Avat3r, can transform these snapshots into an animatable 3D head in minutes. Creating high-quality digital doubles used to require expensive studio setups with multiple cameras and heavy computational power. Avat3r aims to democratize this process by drastically reducing both the hardware requirements and processing time.
The secret sauce lies in making Large Reconstruction Models (LRMs) animatable. These models learn comprehensive 3D representations of human heads from extensive multi-view video datasets. To enhance reconstruction quality, Avat3r combines position maps from DUSt3R with generalized feature maps from Sapiens, a foundation model designed specifically for human representation.
The system is surprisingly robust against common user errors. Thanks to training on various facial expressions, it can handle inconsistent inputs like slight movements during photo capture or frames pulled from video.
Are the codec avatars getting closer?
This development fits into Meta's broader vision for photorealistic codec avatars, a technology they've been refining for years. Recent job postings suggest they're ramping up these efforts significantly.
The company is currently recruiting iOS developers with expertise in user interface design and tools for iPhone and iPad applications. This hints at the possible development of a consumer app that would guide users through the avatar scanning process and enable XR communication through future Quest headsets.
Adding credibility to this theory, VR enthusiast Luna has discovered strings in the Quest firmware that reference codec avatar implementation. However, the timeline and compatible devices remain unclear. Currently, only the Meta Quest Pro supports face and eye tracking — other devices like the Quest 3 would need to approximate facial expressions using alternative inputs like voice.
The development of Avat3r represents another step toward making photorealistic digital avatars accessible to everyday users, though we'll have to wait and see how Meta integrates this technology into their consumer products.
What do you think of smartphone avatars? Join the conversation on Facebook, Bluesky or X or share your opinion in the comments below.
For feedback, topic suggestions, or other ideas, please email us at hello@mixed-news.com.
Note: Links to online stores in articles can be so-called affiliate links. If you buy through this link, MIXED receives a commission from the provider. For you the price does not change.
