A USER’S GUIDE TO A NEW YET ANCIENT ART FORM.
Ok, I admit claiming Virtual Reality is the final frontier is a cheap attention-seeking ploy and I risk being guilty of false advertising. Before you send me to the sin bin along with Billy McFarland and the rest of the Fyre Festival crew, let me explain why Virtual Reality and 360° films are here to stay and given the right situation have the potential to revolutionise storytelling.
The single user nature of headset-based viewing to some degree limits the widespread uptake VR needs to go mainstream, however, there are situations where no other medium comes close to delivering a comparable experience or communicating in the way a 360° film does. The immersive nature of a headset also ensures a far greater level of empathy is felt by the user while within the experience. The recently delivered Krait 360° VR experience Luscious produced for the National Museum of Singapore and the ANMM is a great example of this.
Late in 2018 Luscious produced a second film to join our first and screen in the Action Stations Theatrette at the Australian National Maritime Museum. It will also soon begin screening at the National Museum of Singapore. In Sydney the film is supported by the physical presence of the Krait which is moored in Darling Harbour. Despite this, as an 80-year-old relic it is not possible for the public to go onboard the ship. In Singapore the film only has photographic support. With this in mind, a guided immersive Virtual tour entitled ‘The Krait” has been produced and installed at both the Singapore National Museum and the Australian National Maritime Museum.
Luscious has produced several VR projects and the Krait experience stands out as a step forward for 360° film storytelling.
A common concern with 360° and VR is a perceived inability for the film makers to control the viewers gaze. While there are challenges, this isn’t true and with a rethink it is possible to unleash the potential of storytelling in 360°. Sure, if someone wants to stare at the attractive chorus member at the back during Phantom of the Opera they certainly can, but if you approach 360° in the similar way to how a director approaches theatre, it is possible to create a ‘guided’ 360° experience with most viewers seen to repeat similar head movements while viewing the experience.
NB: We believe there is a clear difference between 3 Dof 360° VR and 6 Dof VR as in 6 Dof the viewer has far more control over their journey. We don’t see 360° VR as virtual reality, it is 360° film viewed through a VR headset.
To create a successful 360° film, you must first build or choreograph an environment which encourages exploration and allows a story to emerge through a process of user-driven yet film-maker guided discovery and this shift represents a seismic change from traditional filmmaking, where skills in photography and a keen appreciation of fine art tend to drive the expertise of the most competent directors. It is our contention that 360° video is more like theatre than film and even more similar to proscenium arch theatre. In proscenium arch theatre an actor may appear on a balcony or in a box causing the audience to turn so as to hear and see what is happening. This is very similar to the situation which exists in 360° and VR. A director working in 360° or VR has all the tools available to a theatre director, who using lighting, sound, blocking and dialogue to guide the audience gaze. It is also possible to use spatial sound techniques and traditional film framing to ensure the audience is concentrating on the right part of the story as you would in traditional film. Even with a lens system which captures 360°, it is still possible to use elements such as windows, doorways, movement and structure to influence the viewer’s attention. By placing interesting objects near or far relative to the lens, it is also possible to create a sense of depth and guide the tracking of the viewer’s attention.
There is evidence 360° film and VR can inspire empathy in viewers. They allows creators to truly affect their audience and the stronger the connection a viewer has to a story, the more memorable it becomes. For example, while filming the Krait it became apparent it was unnecessary to state how cramped living conditions were aboard the ship, as the spatial view through the VR headset made this abundantly clear.
While we designed the Krait experience to guide the viewer’s gaze, they still have the ability to experience the story uniquely.
Other than when the experience begins at a wharf, the boat is always at sea and in motion. While not part of the initial brief, 360° or VR is well suited to any experience which involves movement. The spatial quality of the format ensures movement delivers a far more visceral and believable user experience.
It is worth remembering the audience experiences the story through the eyes of the camera and accordingly, where it is positioned in a scene will define how they will interpret the narrative.
A common misconception of 360° and VR is the audience will look at everything around them. This is by no means the case and we believe it is essential to continually ask where we want the audience to look, when we want them to look there and how we want them to react to the what they are seeing and hearing. We believe there is real potential to sweep the audience up in the magic of the story and feel as if they are controlling it. We call this ‘The Illusion of Choice’ because while the audience has the belief they can look anywhere they wish (and indeed they can) if we have done our job as storytellers well, they will look where when we want them to and explore the world as we intend. This illusion of choice is what creates the guided user experience.
We begin the design and production process with a script and record a guide voice followed by a storyboard which we combine to deliver a shooting guide. Using a low cost 360° camera system we spend a couple of hours shooting a rough animatic to match the storyboard, which when edited together creates an approximation of the finished experience. This process offers the client a degree of comfort and enables us to better predict design where we want the viewers to look and for how long they look there when watching the finished film.
Before the shoot and using the rough edit as a guide, we set elements such as camera position, framing, lighting and sound so as to define the user experience and control the gaze. With the edit timed to suit we designed animated graphic overlays in 3D space to add additional information and further influence the audience gaze while on the virtual tour of the boat.
With the need to predict the viewer’s gaze direction at each edit point, longer render times and the requirement to work with rectilinear images creating a sense of flying slightly blind, the post production process is certainly more challenging. However, once you have mastered these factors, the principals of digital design are the similar to when working on a non-spatial project (be it a somewhat complex production).
We installed the experience on a collection of Oculus Go headsets which were programmed to operate in Kiosk mode. Reprogramming the user interface into kiosk mode is clearly not something Oculus intended for their headsets and achieving and it required a determined effort by our UX programmer. Delivering an easy to use experience was essential as the risk of user confusion quickly rises if someone puts the headset on then struggles to navigate the menu. We designed the controls to be influenced by where the user looks (gaze based) and as simple as possible. This includes having the film momentarily pause if the user takes the headset off, then presenting them with two gaze selected options to restart or continue if they put the headset back on.
While some filmmakers have the impression good craft and storytelling cannot be achieved in 360° video and VR, we respectfully disagree and believe the ease of use, level of enjoyment and uniqueness of the Krait experience is a testament to this. It is entirely possible to create great films in 360° and VR. While by no means suitable for all projects, when matched with the right production 360° and VR are capable of delivering immersive and empathy building experiences like never before.
Thanks to the dedicated staff at the Australian National Maritime Museum who helped make the Krait 360° VR experience possible, especially Michael Harvey, Richard Wood, James Hunter, Stephen Crane, David Payne, Vanessa Weedon and the crew of Krait. Special thanks to Priscilla Chua of the National Museum of Singapore.
If you want to view the Krait 360° VR experience or other Luscious 360° VR or Virtual Reality productions contact Richard or Jeff at www.luscious.com.au/contacts/