visionOS 2 for Apple Vision Pro is here and lets you convert 2D images to Spatial Photos
The first major update to the spatial operating system brings new features to Apple's headset, including customizable Home Views and the ability to convert 2D photos into spatial photos.
Apple has released visionOS 2 for the Apple Vision Pro. The update implements the new features and improvements for the spatial operating system announced at WWDC 24. We have summarized the most important features here.
Content
Spatial photo conversion and SharePlay support
With visionOS 2, you can now convert traditional 2D photos from the Photos app into spatial photos. According to Apple, machine learning is used to add natural-looking depth to images.
The Photos app also gets SharePlay support. This allows you to share content such as photos, panoramas, and Spatial Videos taken with the new iPhone 16 and iPhone 16 Pro with friends and family.
You can also use SharePlay to connect with other users as Spatial Personas and view photos and videos together.
Optimized hand and room tracking, new hand gestures
Apple has also been working on new hand gestures for navigation in Vision Pro. You can now quickly access Home View by lifting and tapping your fingers in front of your face. By rotating your hand, you can get information like the time, battery status, and volume, and open the Control Center.
VisionOS 2 now shows hand tracking more smoothly and at the refresh rate of the display. This should be 90Hz by default.
Apple is also introducing Prediction, an optional feature for developers. It provides optimized hand poses that indicate where the hand should be at the time each image is displayed. This is intended to reduce latency, although in some cases it may result in less precision.
VisionOS 2 also brings a new room tracking feature to Vision Pro, allowing developers to better capture the geometry of rooms.
Video: Apple
Enhanced mouse and keyboard compatibility
VisionOS 2 also brings productivity improvements. You can now use a mouse, which should improve navigation within documents, for example.
In fully immersive environments, Vision Pro also automatically displays a view of the physical Magic Keyboard.
The Home View can now be personalized by freely arranging applications.
Other new features of visionOS 2
- Trains are now also available in travel mode
- streaming of content from iPhone, iPad, or Mac now works via AirPlay on the Vision Pro
- System-wide live subtitles for conversations and audio content
- Guest mode saves the eye and hand data of the last guest for 30 days to enable faster access.
- Panorama feature and Cinematic Mode for Safari: You can now watch videos from Netflix or YouTube in the browser on huge virtual screens and in any environment.
- WebXR is now activated by default
- New immersive environment: the island paradise of Bora Bora
- Personas get more accurate skin tones, vibrant clothing colors and smoother, more natural hand movements
- You can now add different backgrounds from third-party providers during FaceTime or video calls
- Mindfulness app gets new feature called "Follow your breathing"
- Live Captions supports you with real-time transcriptions of voice, audio and video content
The following features will be released later this year
- Multiview support for the Apple TV app
- Mac Virtual Display gets a customizable, extra-wide screen that expands around you.
Video: Apple
New developer frameworks
Apple announced visionOS 2 at the WWDC developer conference in June. According to the company, there are now more than 2,000 native apps for Vision Pro and 1.5 million compatible iOS and iPad apps. To accelerate the development of further apps, Apple also presented new interfaces and frameworks at WWDC, which are now also available with the release of visionOS 2.
These include Volumetric APIs, which allow multiple complex 3D apps to run simultaneously, TabletopKit for creating apps that can be anchored to tables and shared with others, and Enterprise APIs for enterprise apps.
Note: Links to online stores in articles can be so-called affiliate links. If you buy through this link, MIXED receives a commission from the provider. For you the price does not change.
