VR: THE FINAL FRONTIER.

A USER’S GUIDE TO A NEW YET ANCIENT ART FORM.

The_Final_Frontier.png

Ok, I admit claiming Virtual Reality is the final frontier is a cheap attention-seeking ploy and I risk being guilty of false advertising. Before you send me to the sin bin along with Billy McFarland and the rest of the Fyre Festival crew, let me explain why Virtual Reality and 360° films are here to stay and given the right situation have the potential to revolutionise storytelling.

The single user nature of headset-based viewing to some degree limits the widespread uptake VR needs to go mainstream, however, there are situations where no other medium comes close to delivering a comparable experience or communicating in the way a 360° film does. The immersive nature of a headset also ensures a far greater level of empathy is felt by the user while within the experience. The recently delivered Krait 360° VR experience Luscious produced for the National Museum of Singapore and the ANMM is a great example of this.

Late in 2018 Luscious produced a second film to join our first and screen in the Action Stations Theatrette at the Australian National Maritime Museum. It will also soon begin screening at the National Museum of Singapore. In Sydney the film is supported by the physical presence of the Krait which is moored in Darling Harbour. Despite this, as an 80-year-old relic it is not possible for the public to go onboard the ship. In Singapore the film only has photographic support.  With this in mind, a guided immersive Virtual tour entitled ‘The Krait” has been produced and installed at both the Singapore National Museum and the Australian National Maritime Museum.

Screen Shot 2019-01-30 at 11.25.49 am.jpg

Luscious has produced several VR projects and the Krait experience stands out as a step forward for 360° film storytelling.

A common concern with 360° and VR is a perceived inability for the film makers to control the viewers gaze. While there are challenges, this isn’t true and with a rethink it is possible to unleash the potential of storytelling in 360°. Sure, if someone wants to stare at the attractive chorus member at the back during Phantom of the Opera they certainly can, but if you approach 360° in the similar way to how a director approaches theatre, it is possible to create a ‘guided’ 360° experience with most viewers seen to repeat similar head movements while viewing the experience.


NB: We believe there is a clear difference between 3 Dof 360° VR and 6 Dof VR as in 6 Dof the viewer has far more control over their journey. We don’t see 360° VR as virtual reality, it is 360° film viewed through a VR headset.

1.jpg

To create a successful 360° film, you must first build or choreograph an environment which encourages exploration and allows a story to emerge through a process of user-driven yet film-maker guided discovery and this shift represents a seismic change from traditional filmmaking, where skills in photography and a keen appreciation of fine art tend to drive the expertise of the most competent directors. It is our contention that 360° video is more like theatre than film and even more similar to proscenium arch theatre. In proscenium arch theatre an actor may appear on a balcony or in a box causing the audience to turn so as to hear and see what is happening. This is very similar to the situation which exists in 360° and VR. A director working in 360° or VR has all the tools available to a theatre director, who using lighting, sound, blocking and dialogue to guide the audience gaze. It is also possible to use spatial sound techniques and traditional film framing to ensure the audience is concentrating on the right part of the story as you would in traditional film. Even with a lens system which captures 360°, it is still possible to use elements such as windows, doorways, movement and structure to influence the viewer’s attention. By placing interesting objects near or far relative to the lens, it is also possible to create a sense of depth and guide the tracking of the viewer’s attention.

2.jpg

There is evidence 360° film and VR can inspire empathy in viewers. They allows creators to truly affect their audience and the stronger the connection a viewer has to a story, the more memorable it becomes. For example, while filming the Krait it became apparent it was unnecessary to state how cramped living conditions were aboard the ship, as the spatial view through the VR headset made this abundantly clear.

3.jpg

While we designed the Krait experience to guide the viewer’s gaze, they still have the ability to experience the story uniquely.

Other than when the experience begins at a wharf, the boat is always at sea and in motion. While not part of the initial brief, 360° or VR is well suited to any experience which involves movement. The spatial quality of the format ensures movement delivers a far more visceral and believable user experience.

It is worth remembering the audience experiences the story through the eyes of the camera and accordingly, where it is positioned in a scene will define how they will interpret the narrative.

A common misconception of 360° and VR is the audience will look at everything around them. This is by no means the case and we believe it is essential to continually ask where we want the audience to look, when we want them to look there and how we want them to react to the what they are seeing and hearing. We believe there is real potential to sweep the audience up in the magic of the story and feel as if they are controlling it. We call this ‘The Illusion of Choice’ because while the audience has the belief they can look anywhere they wish (and indeed they can) if we have done our job as storytellers well, they will look where when we want them to and explore the world as we intend. This illusion of choice is what creates the guided user experience.

We begin the design and production process with a script and record a guide voice followed by a storyboard which we combine to deliver a shooting guide. Using a low cost 360° camera system we spend a couple of hours shooting a rough animatic to match the storyboard, which when edited together creates an approximation of the finished experience. This process offers the client a degree of comfort and enables us to better predict design where we want the viewers to look and for how long they look there when watching the finished film.

Before the shoot and using the rough edit as a guide, we set elements such as camera position, framing, lighting and sound so as to define the user experience and control the gaze. With the edit timed to suit we designed animated graphic overlays in 3D space to add additional information and further influence the audience gaze while on the virtual tour of the boat.

Screen Shot 2019-01-30 at 11.26.05 am.jpg

With the need to predict the viewer’s gaze direction at each edit point, longer render times and the requirement to work with rectilinear images creating a sense of flying slightly blind, the post production process is certainly more challenging. However, once you have mastered these factors, the principals of digital design are the similar to when working on a non-spatial project (be it a somewhat complex production).  

We installed the experience on a collection of Oculus Go headsets which were programmed to operate in Kiosk mode. Reprogramming the user interface into kiosk mode is clearly not something Oculus intended for their headsets and achieving and it required a determined effort by our UX programmer. Delivering an easy to use experience was essential as the risk of user confusion quickly rises if someone puts the headset on then struggles to navigate the menu. We designed the controls to be influenced by where the user looks (gaze based) and as simple as possible. This includes having the film momentarily pause if the user takes the headset off, then presenting them with two gaze selected options to restart or continue if they put the headset back on.

Screen Shot 2019-01-30 at 11.28.17 am.jpg

While some filmmakers have the impression good craft and storytelling cannot be achieved in 360° video and VR, we respectfully disagree and believe the ease of use, level of enjoyment and uniqueness of the Krait experience is a testament to this. It is entirely possible to create great films in 360° and VR. While by no means suitable for all projects, when matched with the right production 360° and VR are capable of delivering immersive and empathy building experiences like never before.

Thanks to the dedicated staff at the Australian National Maritime Museum who helped make the Krait 360° VR experience possible, especially Michael Harvey, Richard Wood, James Hunter, Stephen Crane, David Payne, Vanessa Weedon and the crew of Krait. Special thanks to Priscilla Chua of the National Museum of Singapore.

If you want to view the Krait 360° VR experience or other Luscious 360° VR or Virtual Reality productions contact Richard or Jeff at www.luscious.com.au/contacts/

ANMM - OPERATION JAYWICK – DARK VICTORY

Luscious recently produced a second film to feature in Action Stations at the Australian National Maritime Museum. Entitled ‘Operation Jaywick – Dark Victory’ the film was a joint production between the Singapore National Museum and the Australian National Museum showcasing the birth of Special Forces in Australia.

Due to the covert nature of the original operation there was a very limited amount of historical material to tell the story. The 16 metre by 3 metre ultra-widescreen format of the delivery dictated the images were built in 6K requiring a technically complex production and delivering workflow to enable us to meet the delivery date working to a compressed timeline.

Luscious undertook a significant amount of historical research before writing the scripts, storyboarding the film, researching and creating the 2D and 3D imagery (including a true-to-life 3D CGI Krait and the foldable canoes) then producing and delivering the critically well received film.

Luscious commissioned, briefed and produced the soundtrack including sourcing historical sound bites and historically accurate sound-bite recreations which were brought together to create a collage style narration for the film.

 Informational Graphics were designed and projected into 3D space with 2D subtitles then added in several languages to accommodate the international nature of the audience.

A guided immersive Virtual tour of ‘The Krait” is now under production with the look and design matched to the original ‘Operation Jaywick – Dark Victory’ film. Upon completion the experience will be installed at both the Singapore National Museum and the Australian National Maritime Museum.

JAMES CAMERON - CHALLENGING THE DEEP

Luscious was commissioned by the Australian National Maritime Museum to create an immersive, narrative-rich museum experience using emergent technologies to document James Cameron’s work as an explorer and storyteller. 

The exhibition features five films playing on four large screens set so as to be enjoyed as a linear or non-linear journey within the exhibition space. The soundtrack and the computer controlled programming of the films is designed to deliver narrative specific relative to the location of the visitors throughout the gallery.

As well as film scripting, production and editing, soundtrack conception, interactive display design and the computer controlled lighting plan, Luscious created the television, online and print advertising campaign to promote the exhibit.

James Cameron; Challenging the Deep, opened its doors on the 28th of May to critical and public acclaim with global interest building in what is truly transformative experience.

IPP - SHREWSBURY MIXED REALITY PET STORE

IPP America commissioned Luscious to create a life-like 6 degrees of freedom experience within a fully stocked and visually operational pet shop.

A combination of filmed live-action, digital matte paintings and 3D CGI were combined to deliver the 6 Dof immersive experience. 

The structure of the store was built from 2 dimensional CAD file plans of a real store then texture mapped using colours, signage and art work created by the advertising agency.

Real staff were filmed in an operational store and the footage was then looped to create a continuous visual narrative. 

The sound recorded on location was processed in Unreal Engine to create 3 dimensional location specific spatial sound.

A high level of realistic detail was created using 3D CGI stock texture mapped with actual products from the store.

It would be feasible to create an operational virtual store where customers could inspect and buy products online for later delivery.

VIRTUAL EQUALITY – VIRTUAL REALITY ADVERTISING CAMPAIGN

Luscious produced the Cannes Silver Lion award winning virtual reality wedding campaign for J. Walter Thompson Worldwide.  

We filmed a wedding in Auckland New Zealand in 360 VR then created a multi-view VR experience, which was later enjoyed by guests at a virtual wedding in Sydney where same sex marriage was not yet legal. 

The Virtual wedding experience in Sydney was filmed then edited to create an online advertising campaign with the powerful message: "Australia, It’s time to make marriage equality a reality.” Then becoming a rallying cry of the “yes” campaign.

This highly successful approach of producing a VR or AR component which then serves as the base component for a commercial which can be filmed and screened in a traditional manner proved to be a highly successful and repeatable approach to creating an advertising campaign.