Virtual Arena: Augmenting the Entertainment Playscape!

The Application of XR into the attractions and amusement landscape is covered by industry specialist Kevin Williams, in his latest Virtual Arena column – we look at the deployment of Augmented Reality (AR) and Mixed Reality (MR) technology into the Out-of-Home landscape.

Author: Kevin Williams

In this latest column for MIXED, we look at the application of AR in the Out-of-Home entertainment market, as defined by the two approaches to the application. First, we will look at those employing optical “See-Through” immersion, via screens or glasses.

History of AR

The oldest utilisation in entertainment, was seen with the audience experience at the theater of the 1800s, revolutionized by the adoption of “Pepper’s Ghost” – a semi silvered sheet of glass placed in front of the stage. Correctly illuminated, it allowed ghostly apparitions to be presented as if on stage to the viewer – first used in 1862. An application long before the digital revolution that would see a second life in fighter planes, incorporated into gunsights – and eventually evolving into the more advanced Heads-Up-Display (HUD).

But it would be with the digital age, and the experimentation in wearable technology that we would see, in 1966, Ivan Sutherland demonstrates the “Sword of Damocles”, and the concept of general-purpose computing that would evolve into “Spatial Computing”. Manipulating synthetic objects represented into the real-world. The experimental “Telesphere Mask” would become known as the “Sword of Damocles” due to the weight relief system needing to address its bulk.

Beyond greater military utilization of this technology, it would not be until commercial entertainment got its hands on the environment that it would see wider consumer recognition as a brand-new interactive display medium.

Application of See-Through AR

One of the first Japanese amusement factories that dabbled with AR amusement attractions was BANDAI NAMCO – having worked on several VR arcade projects. The company moved into AR, developing a pop-up attraction for their short-lived ‘VR ZONE’ venues, called ‘PAC In Town’ (2018). This saw the deployment of the original Microsoft Hololens, one of the first times the unit was used in an amusement setting.

Players ran around a digital maze, seeing virtual representations of PACMAN and the ghosts, represented by each player, using the AR headset. The system being a pop-up installation only operated at two Japanese locations for a short period, as this and other amusement manufacturers evaluated the audience reaction to this immersive tech.

Jump forward to the current entertainment sector, and we see AR being applied in both see-through and pass-through technology. Mixed reality applications with a strong AR element gaining interest in a market looking for the next major draw – following on from the previous generation of VR technology, MR seems to be shaping up as the next big trend this market will be promoting.

Examples of MR “immersive attractions” with guests walking within digitally populated environments being developed. Many of these using the Ximmerse see-through AR headsets.

Examples also include Chinese developers who have mixed AR technology with electric karts, launching their own ‘MR Karting’. Racers wear see-through AR headsets, steering their vehicle round the physical course, avoiding fellow racers, and collecting virtual objects.

This example developed by VCAN Technology offers a children’s experience with animated characters moving within the space. The concept has also been developed with a tablet mounted on the kart offering a view of the real-world and virtual objects.

Holographic Theater Experiences

The themed experience or immersive attraction approach to AR deployment has gathered momentum, as seen from Enklu. Developer of what they call “Holographic Theater”, they are famous for their creation of the ‘Unreal Garden’. An augmented playscape, first opened in San Francisco, as part of the short-lived ‘OneDome’, this multi-player mixed reality experience immersive users within magical surrounds populated with fantastical flora and fauna – both physical and digital. The guests navigated through the garden, interacting with the elements using their AR glasses and gesture tracking. Enklu became a Microsoft Mixed Reality Partner in 2017, and their installation fielded the Microsoft Hololen’s AR headset the following year.

The company has gone on to deploy the experience as a pop-up attraction – seen as part of the E3 2019 exhibits. And after an update to ‘Unreal Garden 2.0’, was installed in a new standalone location as Verse Immersive San Francisco, part of the establishment of their unique venue approach. The attraction now reverting to using the Microsoft Hololens 2 headset, with a haptic element added to the attraction using Ultraleap.

Enklu, announced a new pop-up installation that will open at Boxi Park at Lake Nona, in Orlando, Florida. A giant purpose-built space for the Verse Immersive Orlando venue. Offering the brand-new experience ‘The Unreal Garden: Escape the Metaverse’ – as a holographic dreamworld featuring art by Android Jones. All part of the immersive experience, working with operator Fever. This seen as an approach to “Artainment” installations that are springing up internationally, building on AR immersion theater.

Expanding the number of immersive AR experiences from Enklu, the operation recently launched ‘Sightcraft’ – a magical encounter, that allows the audience to experience a game that blends spell casting and social interaction through the AR technology – again using the enterprise based Hololens 2 platform. The team having already opened Verse Immersive Chicago and Denver, building on this business formula.

We have seen live-experiences and theater performances deploy see-through AR – mainly as a viewing experience, as well as a full interactive element. We have attended in 2021 the opening of ‘The Lost Origin’. A collaboration between immersive entertainment studio Factory 42, the Almeida Theatre, Sky, and the University of Exeter. A proof of concept that, along with live performance and audience puzzle solving, deployed Magic Leap ML1 headsets. The experiences including volumetric capture bringing MR into live performance.

Most recently, a New York theater opened this year a new performance called ‘KAGAMI’ – that blended live performance, film, music and MR, in a 15-minute experience. The audience of 80, using the new enterprise-focused Magic Leap ML2 headset to offer AR elements to the performance.

Several corporate pop-up promotions have been deployed using the ML1 consumer system and other headsets, in unique AR marketing promotions. As seen with Sony Japan’s 2019 Ginza Park ‘Ghostbusters Rookie Training: AR Experience’ demo debut that promoted the movie property anniversary and fielded their prototype AR headset as a commercial entertainment solution, with haptic wristbands.

The Theme Park Dimension

The Theme Park sector has not been slow to embrace AR technology as a new immersive tool in their attraction development armoury. But regarding interactive deployment of AR as a larger theme park ride attraction can trace its roots back to the 1990s. As we have mentioned before in previous features on this sector, one of the touchstones for immersive technology in a destination setting was the work done by amusement developers. Developers like SEGA, with their ambitious “Mid-Scale Attractions” for their idea of Amusement Theme Parks (ATP).

One such innovative concept was ‘Ghost Hunters’ (1994) – a dark ride attraction which incorporated interactivity and the use of an AR element – players shooting at CGi ghosts, created by appearing on the ride vehicle screen as if in the scenery show-set (Pepper’s Ghost for the digital age). This unique mid-scale attraction was deployed in several SEGA ATP venues during the period, offering a tantalizing glimpse of what this technology could achieve with greater graphical performance.

SEGA would also develop one of the first pass-through augmented reality attractions, even further back in 1988 with their ‘SEGA Super Circuit (SCC)’. In a foreshadowing of Nintendo’s ‘Mario Kart Live’ some 32-years later, but on a grandiose scale.

Five radio control cars, steered from modified arcade driving cabinets, relaying the first-person view of the miniature racetrack via CCTV camera’s mounted in the vehicles. This pass-through mixed reality experience proving incredibly popular, if not a technical burden, (based on a previous exhibit). Even today, several manufacturers have launched modern examples of this amusement AR piece.

Currently, interactive AR applications in the theme park sector have received a massive shot in the arm with the partnership between Universal Studios and Nintendo, towards creating ‘Super Nintendo World’. A new park-gate, originally for Japan’s Universal Studio theme park in 2021, but then opened in Hollywood, California (2022) and soon in Orlando, Florida, in 2024. Part of this ambitious playground modelled on the digital game world of Nintendo is the innovative attraction ‘Mario Kart: Koopa’s (Bowser’s) Challenge’. Fundamentally a dark ride, with four player ride vehicles offering a 5-minute AR experience.

It is the unique immersive and gamified elements of the attraction that makes it so special. Riders wear a unique two-part headset, based on the distinctive Mario cap, with an AR visor. And during the attraction, see digital elements interposed with the show-set, collecting coins during the ride, from items they look-at and hit that go towards their score, stored on their profile. The AR element is based on the headset developed by Mira – a specialist in commercial and military AR head-mounted-displays (HMD). Interest in their AR applications was such that Apple acquired Mira in June of this year.

Application of Pass-Through AR

The second deployment of AR technology is through video “Pass-Through” immersion, via cameras. This application defined as a Mixed Reality (MR) approach that in some applications combines AR and VR elements.

One of the most publicly recognized deployments of what AR can offer has been from Niantic, and their incredibly popular ‘Pokémon Go!’ title. A location-based Pokémon scavenger hunt mobile game app that saw over a billion downloads and became its own phenomena. However, the application fell afoul of its difficulty to police the movement of the players, with numerous incidents. Illustrating the benefit of applying this tech in a controlled location-based entertainment space.

Attractions developers like Holovis have applied the scavenger hunt style of AR experience for guest’s smartphones into the theme park arena, with their 2017 installation of ‘Battle for Kings Dominion’ at Cedar Fair.

Guests navigating in this parkwide mobile game experience collecting points from achievements made ranging from rides taken, mini-games played, and areas ventured into.

Virtual Sports Explodes

Regarding standalone attractions employing interactive AR, and we have seen the successful deployment of ‘HADO’. Developed and operated by meleap – the game has been described as a new “techno sport”. Comprising teams of players wearing the latest smartphone enabled mixed reality headset, in an AR-game.

A highly active Player-Vs-Player (PVP) combative game experience, launching virtual fireballs (“energy orbs”) at opponents, while dodging and using virtual shields in defence. The experience is a frenetic and high-speed adventure, as fun to watch as to play.

First started in Japan in 2014, meleap has grown ‘HADO’ into a championship experience, with tournaments’ run and even streamed coverage of events. The experience has morphed into a full eSports competition, surpassing many of the PC and VR streamed events – with their own dedicated channel.

Already, the operation has opened ‘HADO Arenas’ across Japan, and in 39 other countries across the rest of the globe. Because of the target audience of school age players, and its eSports credentials, the UK HADO operation has partnered with ESL FACEIT Group, (the eSports gaming championship community). This AR experience becoming one of the fastest growing virtual sports.

The Japanese company has also placed investment into other applications of the interactive AR technology, creating ‘HADO Monster Battle’ – a team-based monster combat arena for younger players. A game experience building on the popularity of their PVP title, but with a new fantasy and fast shooting narrative.

The latest applications of the meleap platform utilizing two devices, one being a customised smartphone AR headset, and the other a smartphone wrist mounted interface for controlling the action.

Another developer of multi-player PVP combative AR experiences for location-based entertainment are from BattleVerse. Having created a compelling combative experience of their own, that they have licensed to other operators wanting to include a team-based experience at their entertainment facility.

The unique elements of this approach to AR are much more suited to multi-player experiences than the more solitary applications of AR gaming seen in consumer deployment.

Coming AR Attractions

The development of compelling standalone AR attractions has seen the creation of ‘Laser Limbo’ – a free-roam AR immersive battle game. Released by FreeRoam.AR – a studio marrying social interaction with immersive reality. The game is laser tag with physical items supported by AR weapons and effects within a mapped environment.

The system has been demonstrated using MR headsets (Meta Quest Pro) and hoping to offer a new level to this style of experience. This new development is partnering with Enklu – as we have mentioned previously, with their Verse Immersion venues and AR holographic theater work. The operations will be working to deploy experiences into entertainment centers and museums.

Many of these new AR developments utilize the guests own mobile device to play. A new entrant into the interactive AR scene is developer ONTOP Studios, who have just launched their AR arena game called ‘New Fantasy’. Players using an AR mobile game app to play, attacking the deity in the center of the arena, dodging and launching their attacks, playing as a team to collect the treasure from the vanquished opponent. The game supported by a tournament through the app, players building the stats for their character.

Not only creating new sports games, but also giving new life to existing entertainment. ‘Puttscape’, is a collaboration between AR and a physical mini-golf course. Created by AR development studio DAS LABS and partnering with mini-golf operator Old Town Putt – the creation of a marriage of physical golfing, with the tracked ball interaction with virtual objects and characters placed virtually onto the course – viewed through wearing a MR headset. Creating a dynamic augmented solution where the game can change being first space themed and then pirate, with the virtual elements adding animations and scoring to the game. This is one of the games supporting Niantic’s ‘Lightscape’ platform, to support AR development in the real world.

The Next Phase of Adoption

The application of AR has received even greater media interest following Apple’s big reveal of their ‘Vision Pro’ platform – a productivity prosumer Mixed Reality first-generation system. The professional platform a devkit for AR applications to work within the visionOS ecosystem – all planned to be available at the beginning of 2024. Launched during the developer conference, Apple will be looking for applications for their platform and commercial Out-of-Home entertainment will be bound to be on the menu, even for such an expensive system.

The Apple Vision Pro, speculated to sell for $3,499 on launch, is expensive, but in comparison to MR enterprise platforms, is in a similar price bracket. For example, the enterprise ‘Magic Leap ML1’ system (2018), sold for $2,300, while the ‘Magic Leap ML2’ system (2022), sells now for $3,299. The ‘Ximmerse Rhino X Pro’ headset (2022) sells for $3,999, with the ‘Microsoft Hololens’ headset (2016) sold for $3,000, while the ‘Microsoft Hololens 2’ system (2019) was being sold for $3,349. These pricings are not an issue for deployment in commercial entertainment, needing to have the best, as much as a concern for consumer buyers.

As we have mentioned, Apple has continued to amass their AR holding, acquiring the instrumental technology provider from the Mario Kart attraction, Mira. While we see Niantic launching their ‘Lightship’ ecosystem (with expected glasses), and Meta looking to redefine MR with their new Meta Quest 3 launch. Though, most LBE applications will pivot more towards new enterprise capable headsets that are entering the sector from manufacturers such as HTC, Lenovo, Canon and Samsung.

Interest in AR / MR / XR applications accelerated with new developments, and the LBE AR landscape looks to be one of the first battlegrounds for market domination. More coverage of the developments in this scene, following soon.

Note: Links to online stores in articles can be so-called affiliate links. If you buy through this link, MIXED receives a commission from the provider. For you the price does not change.