Sundance, SIGGRAPH, and What’s Next for VR Filmmaking

by Maxwell Planck | Feb 13th, 2019

We are still in a high-fashion stage of VR where the experimentation and far-out art being presented at festivals like Sundance have yet to inspire the “ready to wear” market we are all waiting for. Since 2013, Sundance has been the first annual “runway” that sets the tone for what to expect from VR storytelling in the year to come. This year was no exception, with its New Frontier programming showcasing some of the most pioneering work being created. Festivalgoers were just as excited to see some of the groundbreaking work as they were at the height of VR’s recent resurgence. With limited seating and high interest, all six VR programs — including three VR Cinema programs — were some of the hottest tickets for the 10-day festival. It was paramount that if ticket holders did not show up 15 minutes before showtime, they would lose their spot to someone from the waitlist. Even though consumer VR is suffering from anticipatory malaise, the audience at Sundance still believes something important is going to emerge out of this early experimentation, as do I.

In terms of what this means for our upcoming VR Theater experience at SIGGRAPH 2019, here are four major takeaways from my visit:

Experiencing VR together is an important evolution

The VR Cinema Program exhibitions at Sundance are one-hour experiences that showcase a curated set of 360-degree, experimental films. Each takes place in a dark room where about 40 audience members sit down in swivel chairs, put on Gear VR headsets, and escape into a personal viewing experience. In 2018, it was easy to forget that you were surrounded and I was startled whenever my knee would brush up against my neighbor. This year, a single coordinator would cue the program to begin only after all audience members dawned their gear. I especially felt goosebumps when I was watching a funny or startling moment and heard muffled chuckles and guffaws from my surrounding peers. It heightened my enjoyment and made me feel connected with the strangers around me, making the isolating act of wearing a headset a bit more social.

360-degree cinema remains an interesting format for exploration and experimentation

While experiencing VR at the festival, I saw two interesting new techniques that were added to the medium’s language, which I am calling “zoom teleport” and “spatial subtitling.”

In “Kaiju Confidential,” the director guided the audience through a comedy of two city-sized monsters in the middle of a neurotic and petty quibble over rampaging rights as they smash buildings and swat at pesky fighter jets. As viewers, we watched the quibbling unfold from human scale and kaiju scale, and the director, Ethan Shaftel, moved between these viewpoints with fast zooms, changing both position and scale over the course of, at most, 20 frames. By moving so quickly — and in the direction the viewer is most likely looking — the film avoids simulation sickness and preserves spatial awareness in ways that “blinking teleportation” does not.

In “4 Feet: Blind Date,” a touching film from an Argentinian team that is about a paraplegic looking for sexual satisfaction, the director wanted to preserve the cultural dialogue. Instead of dubbing the film for the festival, they integrated subtitles into the space around the actors. These “spatial subtitles” kept me focused on the interesting parts of the story and integrated comfortably with the flow of the dialogue.

The audience is evolving with the medium

What’s surprising to me is that festivalgoers are still seeing VR for the first time even though it’s been a substantial part of New Frontier programming at Sundance for the past four years. What’s more exciting is that for those audience members who have seen VR, they have a more developed opinion of what they like and what they don’t.

During the early days of VR narratives, most audiences didn’t know what to do with immersion, often treating it like a passive viewer experience and looking straight ahead for the duration. Now, even “beginner” audiences are engaging more with the content, actively looking around, moving their bodies, and reaching out to feel more connected, sometimes even when the experience doesn’t have any hand-based interaction. VR cinema is still an evolving and beautiful format, but it was pieces like the real-time, immersive “Gloomy Eyes”that really excited festivalgoers. In “Gloomy Eyes,” the audience can move around and peer into the dollhouse world of a “Nightmare Before Christmas”-like story. It was a credit to the creators that they designed a piece that guided the audience to move; however, in the past, I’ve seen audiences completely miss expertly crafted and guided experiences because they were still lost in the novelty of the medium. Not only are creators getting better, but audiences are getting more comfortable.

There’s a particular moment in the experience where the two heroes of the story — star-crossed lovers standing about 5 inches in height — are separated by a gap in a bridge that is approximately one hand’s length across. All of the audience members I spoke to felt so immersed that they wanted to help the characters over the gap. More and more, creators are integrating mechanics from games and narrative together to make more interactive pieces. Until now, there was a legitimate fear that audiences wouldn’t understand or want to engage even if given the opportunity. From what I saw this year, it looks like the audience is ready.

Documentaries and experiential pieces are VR’s most effective use

As in high fashion, a few of the experiments presented were interesting attempts but mostly misses in terms of execution and quality, leaving me wondering why the piece needed to have been in VR in the first place.

What did work really well, though, were many of the documentary pieces, the standouts being “Traveling While Black” by Roger Ross Williams, “Marshall from Detroit” by Caleb Slain, and “Ghost Fleet VR” by Lucas Gath and Shannon Service. In “Ghost Fleet VR,” my throat hardened when I experienced the plight of enslaved fishermen as I was put inside the cramped sleeping quarters in the bowels of an old freighter and the narrator talked about the harsh living conditions. The documentary format works so well in VR because a narrator can guide viewers through the story, while the accompanying visuals surround them, offering an intimate emotional context that would feel distant on a flat screen.

Some of the most interesting pieces weren’t narratives at all but purely abstract explorations of mood and experience, my favorites being “Sweet Dreams” by the inspiring and always enigmatic Marshmallow Lazer Feast and “Runnin’” by Reggie Watts and Kiira Benzing. “Sweet Dreams” was a highlight due to the way it incorporated the act of eating and drinking with surreal, interactive visuals to create a novel, synesthetic trip (as I drank a sweet concoction from a straw, it felt like I was actually drinking in my surroundings). “Runnin’” is a hypnotic, musical dance experience that took volumetric capture data from a troupe of dancers and rendered their movement as pixelated shapes that would distort and pulse with the music. Rather than trying to recreate a photorealism that would require a fidelity of volumetric capture that is really hard to do, the creators worked within the constraints and delivered something that was just as evocative as being in a dance party.

Loved this #NewFrontier VR project at #Sundance. RUNNIN’ featuring @ReggieWattspic.twitter.com/vp3Y7es1eL

— VRScout (@VRScout) February 2, 2019

Conclusion: So what does this mean for the SIGGRAPH VR Theater?

Considering these takeaways, I was inspired and reassured that what my team and I have in store for the SIGGRAPH 2019 VR Theateris poised to push the boundaries of VR in the right direction. I am especially excited to see our beautiful VR lobby come together in a way that will go beyond synchronous play, allowing our audiences to experiment with synchronous presence.

My hope is that this summer’s experience tears down assumptions of isolation and gets us closer to the social joy we get when experiencing a traditional movie theater.

The pieces that inspired me the most at Sundance attempted new techniques and played with how interaction could elevate immersion. They also added new verbs to both the 360-degree and real-time formats. I encourage those teams that are not satisfied with the current state of the medium and are passionate to push it farther to submit to the VR Theater program through March 26. It’s the pioneering art that I want to showcase.

We are still in the high fashion stage of VR and I want to make the SIGGRAPH VR Theater one of the most spectacular runways for conference attendees. It’s what attendees see today that will inform the market of the future.