AI and XR: Two technologies extending reality to millions
Our guest author explains how the revolutionary combination of AI and XR is fueling a new era of Mixed Reality, with increased accessibility and advanced capabilities for highly engaging virtual experiences.
Author: David Weinstein, Director of Virtual and Augmented Reality, NVIDIA
The biggest Virtual Reality players are launching their latest Mixed Reality headsets, prioritizing a smaller and lighter design with impressive spatial computing capabilities. As the Extended Reality (XR) ecosystem continues to expand, an ever-widening swath of industries and users is embracing XR technologies, forever changing how people work and play.
Companies ranging from Google to Lowe’s have begun employing Mixed Reality headsets to enable consumers to play games, retail employees to collaborate inside immersive 3D models of stores and healthcare professionals to model the human body for drug discovery and improved surgical planning.
While the movement to adopt Virtual Reality across business- and consumer-focused organizations isn’t new, only now are advancements in AI — such as computer vision, speech AI and AI assistants — allowing lightweight devices to deliver photorealistic immersion, interactivity and seamless Mixed Reality.
Another common denominator for the expansion of Mixed Reality is the OpenXR standard, which lets developers build 3D applications using a common application programming interface to run across a wide range of Extended Reality (XR) devices without the need to add and maintain expensive customizations. This new standard is being rapidly adopted across the ecosystem by HTC, Microsoft, Varjo and dozens more, resulting in faster product development cycles and a flurry of new applications.
The Dynamic Duo: AI and XR
AI and XR go hand in hand as the industry attempts to move from a niche-like 20 million devices sold to putting at least one device on the heads of every adult in a household. AI accelerates rendering to achieve the high resolution and high frame rates required for seamless immersion. Businesses around the world are identifying and tracking real-world objects in Augmented Reality, which provides an easy, accessible medium for interacting with virtual content.
This includes Medicalholodeck, a platform for medical collaboration and teamwork in Virtual Reality. It uses integrated AI and GPU power to stream 3D medical-imaging content to untethered headsets. With the new cloud rendering technology, capabilities like presurgical planning, real-time reconstruction of CT and MRI scans, and medical training on an XR headset are now at doctors’ fingertips.
Hopping into these immersive experiences, surgeons can better prepare for operations by minimizing the time and effort required to import or change hospital environments before beginning intense procedures. In addition, teams can interact and collaborate virtually for training and alignment on surgical procedures. The combined power of AI and XR is set to transform medicine by streamlining training and surgery preparation for global health professionals.
In the consumer realm, everyday activities like watching a nightly sports game may never be the same. 5G Edge-XR, a project led by London-based GPU-accelerated products and services company The GRID Factory, is bringing XR streaming to the sports broadcasting industry.
3D streaming has transformed the sports-viewing experience by displaying game stats and player data on the views of matches themselves, across mobile devices. Fans can watch live-action replays from any angle on a smartphone, tablet, or headset. In this way, businesses are reimagining viewing experiences, and sports games are only the beginning.
A new look, inside and out
People may not have thought VR was poised to take off based on its early days, when headsets were bulky, heavy, and tied to giant workstations with a sixteen-foot cable. That means users were always constrained to specific locations and relatively high costs. Due to these limitations, mainstream VR adoption was modest prior to the introduction of the modern mobile headset. Now, a broader range of users can take advantage of improved comfort and visual appeal.
Today’s headsets boast a lighter, sleeker look, with some offering unbelievably thin lenses. The shift from bulky to compact headset design has enabled consumers to interact in and with their environments for a longer period of time, without heavy strain on their necks. The shortening and integration of the connecting cable within the headset results in a cleaner design. Users can flaunt a more ergonomic, modernized look that’s as stylish as the new features within.
Not only did VR get an updated external look, but it was also redesigned from the inside. From the high quality of the camera to the accurate eye and body tracking, new and improved features offer users immersive experiences that make it easier than ever to collaborate with individuals and teams around the world. AR and MR headsets are also increasingly coming onto the market, with the goal of seamlessly blending the physical and digital worlds. AR and MR headsets are uniquely positioned to be an extension of a user’s personal space and easily integrated into everyday use.
Extending Reality for the future
With AI and XR already acting as a powerful dynamic duo, the next big step will be defined by a wider distribution of stunning XR experiences that include AI assistants to help people create in the virtual world, as well as build and play in the real world. With the help of digital assistants, everyone will be able to tap into generative AI to spontaneously populate their virtual environments with complex 3D models, stunning, expansive environments and even fully articulated, interactive 3D characters.
With generative AI, the opportunities for XR are limitless. The emerging trends of improved XR and powerful AI assistants will power digital experiences and extend to industries like manufacturing, virtual production and more. From cutting the headset cord to taking VR to the cloud, huge strides have been made in the XR industry as its user base continues to expand. No longer constrained to a single dedicated space, users’ collective realities are expanding beyond what’s been previously possible. Whether using XR for e-commerce or investing in it for education, businesses can tap the newest developments in VR, AR and AI to stake their claim in the growing ecosystem.
David Weinstein is the senior director of Extended Reality (XR) at NVIDIA, responsible for managing the company’s XR products and SDKs, and for engaging with partners to deploy them worldwide. Before joining NVIDIA, Dave founded and ran three tech startup companies.
Note: Links to online stores in articles can be so-called affiliate links. If you buy through this link, MIXED receives a commission from the provider. For you the price does not change.
