Why immersive storytelling has come of age

Having spent the past two years directing Apple TV’s ground-breaking immersive live action series, Adventure – which is the highest resolution immersive live action content ever made – I believe I can safely say that the series will transform what audiences can expect from immersive storytelling.

Adventure is proving to audiences, what many immersive writers, directors and creators have known for some time, that this is where entertainment is headed and there is no turning back.

Why do audiences opt for watching TV and film on a rectangular screen (whether TV, tablet or phone) rather than fully immersing yourself in a story the way you can in a virtual reality headset? 

Until now, the answer has been relatively simple – it’s what we know, it’s easy and, while VR is more immersive, the resolution and quality of VR headsets, has not, until now, been good enough to tempt most audiences. So, they have largely remained the remit of gamers.

Adventure takes you on powerful character-driven narratives with some of the world’s leading extreme sports athletes, as they take on some of the most epic challenges of their careers.

The unparalleled resolution of 8K per eye, as well as the comfort and ease of use of Apple’s Vision Pro, means that we are 100% with them, living every emotional beat, personally invested in their efforts. 

In episode 3, Ice Dive, freediver Ant Williams heads beneath the perilous Arctic ice to break the record for swimming the furthest distance under ice on a single breath, while in episode 5, racing driver Laura Hayes is looking to become the first woman to race to the top of the 14000ft Pike’s Peak in under 10 minutes.

Whether on the ice with Ant or in the moments before the flag drops for Laura, you feel entirely present in the scene.

Filming immersive live action content, particularly for the Vision Pro, presents both a unique opportunity but also a handful of challenges.

The series was shot on a proprietary Apple camera in stereoscopic 3D, so two lenses with a 180-degree field of view. 

As an audience member, you have the freedom to look around the scene and, because the resolution is so high, any cinematic movement you want to introduce has to be super smooth or it can lead to motion sickness.

So, we used and developed a myriad of bespoke grip solutions for the series including a telescopic crane atop a 3000ft fjord in ‘Episode 1 Highlining’; the Motion Impossible remote control vehicle in ‘Episode 2 Parkour’; an underwater sled for ‘Episode 3 Ice Dive’ that was perfectly balanced to ensure we could film forward motion with no side-to-side rocking; a vertical cable cam system anchored to the seabed in ‘Episode 4 Deep Water Solo’ and custom built in-car camera rigs from Creokinetics in ‘Episode 5 Hill Climb’, as well as Cinemoves’ high speed camera car. 

The CEO of Atlantic Productions that produced the series, Anthony Geffen, and the director on Episode 1 (who had a key role in developing the series), Elliot Graves, had also spent time working with a German company to develop a close-up rig that could be used in the episodes so that audiences could feel more emotionally connected to our characters. This was something that proved of huge value for the whole series.

Given the need for various grip solutions in each episode, the crew sizes ranged from 35-50 key crew on each episode.  A handful of those crew were consistent across the series, while others changed with the needs of that particular sport and location.

Post-production is more complex in immersive mediums versus 2D. You are dealing with lots more shot detail, the risk of stereo disparity as well as considerations of shot choreography (ie guiding audiences on where to look so key elements of a scene don’t get missed).

Myself and the editor on the series, Rich Lester, spent many hours crafting the episodes that I directed (Parkour, Ice Dive and Hill Climb). During that process, we would be in and out of the VR headset all the time because edit decisions on immersive content feel very different when viewed on a screen in 2D versus when reviewed in headset.

A cut might feel loose on the monitor, but too fast in the headset and, as a result, you loose context and impact. You need the headset reviews to truly find the pace and flow of the scene.

Timing had to be delicately crafted to keep the experience engaging whilst still allowing the audience enough time to absorb the visuals and follow the narrative of story.

If you haven’t tried the Apple Vision Pro, I highly recommend you twist the arm of a friendly Apple employee at a store near you to let you view an episode of Adventure, because I really believe that the potential this opens up for storytellers is limitless. We’re no longer just telling you a story, we’re enabling you to live the story.

Charlotte Mikkelborg is an immersive series director and showrunner