Check out these volumetric photos & videos on your Meta Quest
Two free Meta Quest apps show what the near future of volumetric photos and videos could look like.
Traditional photos and videos have a fixed perspective. So do immersive photos and videos in monoscopic or stereoscopic 180 or 360 formats. As you move your head around the room, the scene follows your movement without giving you a new perspective. This feels unnatural and can even make some users nauseous.
Volumetric media aims to break through this limitation and offer six degrees of freedom in viewing instead of just three: Instead of responding only to head rotation, volumetric photos and videos react to spatial movement, making them more realistic.
One of the problems with true volumetric media is that it is technically difficult to create. It requires a dedicated capture studio with dozens of cameras or huge camera rigs, which is why we haven't seen many good examples in the last decade.
AI could provide the answer
Artificial intelligence has been discussed as a possible solution for easily creating volumetric media. With Gaussian splatting, for example, an environment can be captured relatively quickly in 3D with a smartphone and later viewed in VR. The startup Gracia showed what this could look like on Quest 3.
Scottish startup Cinemersive Labs also uses AI, but with a somewhat less ambitious goal: It has trained an AI model to add three degrees of freedom to traditional photos and videos, allowing you to move your head around the room and view a scene from slightly different perspectives. The advantage is that one day this method will be relatively easy to apply to photos and videos. The disadvantage is that you can only move your head a little in each direction, and otherwise the picture goes black.
The free VR apps Cinemersive Photos and Cinemersive Videoplayer show on Quest headsets just how far this technology can go. That was reason enough for me to take a look at the VR apps.
A subtle but welcome improvement
Cinemersive's examples showed me that you can actually gain a lot by moving your head just an inch or two into the room while watching. The effect is subtle but effective and prevents nausea. The AI models just need to get better to avoid visual glitches.
Artifacts are visible in both photos and videos, and more so in photos than in videos. If you move your head to the side to see what is behind an object, you can see some dark areas and some gaps that are poorly filled by the AI. The videos are more convincing, but you can still see artifacts on the contours of objects.
Despite these shortcomings, I believe in the future of this technology and can easily see it being fully developed in the not-too-distant future. Just as AI can be used today to create photos with a believable depth effect from conventional 2D photos (as Apple does with visionOS 2), it will one day be possible to use AI to add a volumetric dimension to the same photos, within certain motion limits, but without noticeable artifacts.
I highly recommend checking out the VR apps because they give you a taste of the future. And some of the images and videos are impressive to look at in 6DoF.
Note: Links to online stores in articles can be so-called affiliate links. If you buy through this link, MIXED receives a commission from the provider. For you the price does not change.
