Long speculated on and dreamed about in science fiction, virtual reality is finally reality! For filmmakers and film-goers alike, it's re-writing the rules of how we create and consume entertainment. The implications and possibilities for cinema are immense, and exciting. As part of the 65th MIFF, this year we're presenting a special Virtual Reality program, offering a unique opportunity to strap on a VR headset and fully experience the jaw-dropping wonders of total immersive cinema. Next stop: the Holodeck!

Here, Worldview's Steve Spangaro answers a bunch of questions about this new frontier at MIFF, and beyond!

_{Red Dwarf: Gunmen of the Apocalypse. Soon we'll all be able to gunslinging against Death, Famine, War & Conquest!}

With MIFF VR presenting a diverse range of styles in VR storytelling, why the need for a specially produced piece of VR content?

There are great works on offer at MIFF VR and that’s a big part of why we’ve created a VR Intro movie to play immediately before some of them. Many patrons will be trying VR for the first time and the tendency at that point is to get swept away by the uncanny nature of the medium. So the VR Intro is a chance for viewers to get that first moment of wonder out of the way in order to properly enjoy the VR storytelling work they’ve come to see.

You say the MIFF VR Intro is playing in front of ‘some’ of the VR films. Why only some?

That’s more of a technical compatibility thing to do with the synchronous VR playback system created for MIFF. The films Invisible, Jafri, Invasion and The Ark will all be preceded by the VR Intro.

Jafri

Where does this experience take us? How are we adjusting to VR while watching the VR Intro.

Well, we’re most definitely inside The Regent Theatre. And this being VR, I don’t mean we see an interior shot of The Regent interior. We quite literally feel as if we’re inside the Regent Theatre – sitting in the stalls area in seat C38 to be precise.

OK so it’s a film made inside The Regent Theatre? But what happens?

Without giving too much away the viewer gets to see something connecting their immersion with the broader MIFF festival. It's not an overtly narrative style experience but rather a way of being somewhere thematically appropriate that also feels a bit special in the way the location behaves.

Before...

After...

Did you recreate The Regent Theatre as a 3D model or did you actually film it?

We shot video inside The Regent and from that we created both still image and video assets to enable the overall MIFF VR Intro experience.

Can you talk more about what was involved technically.

Well a key element was taking the unconventional path of shooting with a RED camera. RED isn’t commonly used in 360 video production. We also lit the venue with tungsten lights, which presented a challenge because in 360 video you can’t simply hide lights out of shot like you can in flat-screen production.

It’s been a privilege for Worldview to create what will be the entry point to this new medium for a number of MIFF goers. It’s something they may well remember for some time so a lot of work went into making the virtual space feel true to the grandeur and plush tones of The Regent and just generally as realistic as possible.

Setting up...

With an increasing range of 360 video camera systems out there for filmmakers to use, couldn’t you have used one of those?

Not for this project. The Regent has quite dim house lighting and with a conventional array the results wouldn’t have been acceptable. A grainy, noisy picture in VR is just terrible. We don’t see grain in real life so it wasn’t really an option if we were aiming to make the visuals crisp and lifelike.

Also we needed a really high-quality image to work with in post so we could grade very precisely and create some visual effects that used the geometry of the space. The image also needed a lot of clean up on account of equipment for The Sound Of Music being present in the space at the time of shooting.

Did shooting with a RED camera change the way you went about post-production?

Yes and that included handling the deluge of data we’d created.

We’d pretty much maxed out on what is technically possible on the shoot with modern imaging devices, resulting in a master that was 11,500 horizontal pixels. To put that in perspective, IMAX is only about 8,000 horizontal pixels and the increase in file size is exponential as you increase resolution.

We’d pretty much maxed out on what is technically possible with modern imaging devices, resulting in a master that was 11,500 horizontal pixels. To put that in perspective, IMAX is only about 8,000 horizontal pixels and the increase in file size is exponential as you increase resolution.

During one particularly long render, we killed time by calculating that each of our video frames contained 31.5 times as many pixels as you will find in a full HD (1080p) frame. But that’s ultimately what we had to do for people to feel like they are virtually sitting in this beautiful old theatre. In this case it was price of creating what in VR speak is called ‘presence’ – being believably immersed in the setting at the exclusion of everything else.

The Ark

So it’s an ‘immersion’ rather than a ‘film’ as such?

At what what point an immersive experience actually becomes a ‘film’ is one of the many questions being discussed as cinema lovers start looking at VR.

In cinema, a director might take a series of shots or even a whole movie to build an atmosphere. It’s cumulative. But one shot in VR conveys a location and everything that might potentially come with being an observer at that point in space and time. So in VR there’s often more opportunity to let a single shot carry narrative.

A VR shot offers more scope for reflection and speculation. It’s like slow cinema taken as far as it can go, which is really just dialling things back towards the pace at which we naturally experience the world.

Cinema is a language we’ve learned to process and it has always been at arms length from our ‘reality’. If we hurtle viewers through VR spaces in the same way we typically edit scenes in cinema we just freak them out. That’s not how we live life and the closer VR takes us to real life the more we as 360 video makers need to respect our underlying humanity as experienced phenomena.

A VR ‘shot’ offers more scope for reflection and speculation. It’s like slow cinema taken as far as it can go, which is really just dialling things back towards the pace at which we’ve always experienced the world.

Are traditional filmmakers making use of VR?

Most definitely and there are some works at the festival making that clear. Eric Darnell’s Invasion! is a case in point. But by necessity there’s a measure of unlearning to be done before you get started because 360 video isn’t merely an iteration on top of cinema. It’s an entirely new medium.

As a thought experiment it’s interesting to imagine the VR and 360 video medium being created before cinema. We’d then be asking ourselves whether VR experience creators can become good filmmakers!

Is that to say some filmmakers would have trouble moving into VR?

Anyone with a handle on the basics of filmmaking goes through a stage of finding it all very strange and having to consider what tools and insights they can keep and which they must leave behind.

Film and TV has a relatively compartmentalised ‘write it - shoot it - cut it’ thought process that comes under a lot of pressure in 360. The idea is that someone can write a script then someone else can go and scout a location and a crew can turn up and shoot it and any problems that arise will get solved in the edit suite.

Even the most simple, observational documentary content in 360 needs a measure of thinking akin to planning a VFX shot because we don’t have the usual cropping and time shifting tools available to solve problems. We can’t frame up a shot to occlude the things we don’t want shown or, say, use a cutaway shot to solve problems with space and time. Eisenstein’s entire bag of cinematic tricks is pretty much junked by VR.

Eisenstein’s entire bag of cinematic tricks is pretty much junked by VR.

What might be a way around these challenges of moving from cinema to VR?

It’s easy to imagine that question cropping up many times during the VR related Talking Pictures sessions at MIFF, where some leading practitioners will be sharing their thoughts. Attending those talks would be a great place to get some advice before taking the plunge.

Personally, as I work more deeply with the medium I’ve found myself to be ‘designing’ rather than ‘writing’ 360 video ideas. You need more parts of your brain engaged throughout the filmmaking process and, unlike cinema, there is still a long way to go before there is a widely agreed set of conventions that everyone on the crew brings to the set. You have to make stuff up.

It certainly sounds like there’s a lot to think about when making a 360 video.

Yeah, and some days you almost need eyes in the back of your head.

VR at MIFF runs from Tuesday 9 August until Sunday 14 August. The Talking Pictures events are available as part of the VR Symposium ticket.