Every Day Was a Snow Day for Sony Pictures Imageworks and ‘Smallfoot’

VFX supervisor Karl Herbst shares his insights on how his team tackled lots of snow, and a bit of fur, on Warner Bros’ new animated musical comedy adventure.

By Dan Sarto | Friday, October 12, 2018 at 5:14pm

For most of us, the sudden good fortune of a “Snow Day” was indescribably grand. No school. No work. No early alarm clock wakeup. But for Karl Herbst and his fellow Sony Pictures Imageworks artists working on the Warner Bros. animated feature film comedy adventure, Smallfoot, “Snow Day” meant something entirely different. For a film set in a snowbound world above the clouds, practically every day of the production was a Snow Day. Simulating snow. Making snow look fluffy, not wet. Lighting snow. Rendering snow so it didn’t look like cement. Let it snow, let it snow, let it snow!

Herbst, the film’s visual effects supervisor, ultimately had many roles on the production. “At the beginning of the film, I was basically talking to the client about their expectations, so I could figure out how to bid the movie with my producer, Skye Lyons,” he notes. “That turned into wearing a lot of hats and overseeing a lot of people during the main production. I oversaw modeling with my animation supervisor, Kevin Webb. During production, we had CG supervisors to help oversee things. We’re kind of a little army. Kevin took care of animation and rigging. I had a CG supe to oversee surfacing, I had a supervisor overseeing grooming and character effects. I had a supervisor to oversee visual effects, like all the snow in the film, which was quite a lot. And then we built a team of more CG supervisors who become the lighting supervisors on the back side. I supervised everything coming together in the end through lighting out to the final picture.”

From a broad technical standpoint, two of the film’s big production challenges were fur and snow. While the film certainly has a lot of hairy Yetis, as previously alluded to, it has an inordinate amount of snow. For fur, the main issue was determining how many characters, with how much hair, would be onscreen at the same time, which impacted getting manageable render times for desired fur fidelity.

Snow was a much bigger deal. After reviewing how their own studio, as well as others like Disney, have previously produced snow, Imageworks decided the best technical course was to use volumetrics. According to Herbst, “We decided that we didn’t want to use a lot of geometry to actually make snow. In the end, most of our snow shots, especially where there’s a lot of action in the snow, like tunneling, digging, stuff exploding, are actually volumetrics. In essence, it’s a cloud. We chose that route because it allowed us to do things like have snow that sticks together, but then falls apart into pieces which in turn break up into little bits of packed powder. To have characters move through all the different ways snow can form, if you did it with our old method of geometry, it often ended up looking like you’re breaking concrete, or breaking ice, rather than breaking snow.”

“The funny things about snow, that people don’t realize, is that with booming cameras, moves with falling snow ends up looking like rain,” he adds. “The movie had a lot of booming cameras. And of course, the filmmakers never wanted the snow to look like it was wet and heavy. So, we had to develop a system where the snowfall moving at a certain distance out from the camera would still have some gravity to it. That way, when you had a big sweeping camera move, especially a vertical rise, the snow never looked like it turned into rain streaks.”

Learning along the way, Imageworks developed systems to handle other particulars of movement through snow. “We had to develop a system to make sure the snow always looked light and fluffy,” he explains. “We built lots of little systems on top of each other. And then we went back to a simple system of, ‘Hey, we’re going to use displacement maps to make footsteps in snow, but we need to add a little granular stuff to make sure it doesn’t just look like mud that’s getting moved.’ There were a few different systems we had to develop during the production to make it easier to do all this snow.”

The biggest new tech development used on the show was a series of major upgrades on the inhouse Arnold renderer. “We took a big gamble,” Herbst details. “Our dev team told us they had things on their schedule they wanted to add to our Arnold rendering system. They told us our film would really benefit from the enhancements. So, we went into the film knowing that those things were coming. We took that leap of faith.”

“The changes made it easier for our lighting,” he notes. “For ray tracing, noise is always a problem. So, changing to an adaptive sampling scheme rather than a brute force one basically allowed us to create incredibly high-fidelity fur while not making render times impossible for us to manage. It basically sends more rays to areas that need refinement and then says, ‘Hey, this area over here doesn’t need that much. Let’s only send a couple of rays over to this part of the scene.’ So, it automatically selectively managed that for us.”

Additionally, the team made use of what they call Plus-AA, which allowed renders to be stopped and started at any point. “On medium-res renders, if creatively they’re good, and we didn’t have any additional lighting or related notes, we could literally just pick up those renders from where they left off, refine and finish them,” Herbst describes. “That helped us manage noise too because we could collectively say, ‘Hey, because we’re going to defocus this layer of the render in the comp, it doesn’t need to be any more refined. But this other layer does. So, we only need to send that one back to the queue.’ That allowed us to be way more selective about where we spent all of our rendering power and how long it took to actually get the movie done.”

For Herbst, though he spent three years on the project, sprinting to make most of the film in the last year of production, the Imageworks’ team, working closely with the team at Warner Bros., did an outstanding job. “In a big studio environment, trying to produce these films this fast, what normally gets lost is people being inventive on their own. It becomes much more of a checklist of knocking off notes. But on this film, I can’t tell you how many times, in layout, modeling, animation and effects, everybody was coming in with their own ideas and just upping the ante as we went. Really bringing their A game. So, for me, it was incredibly enjoyable going into rounds every day because everyone was so excited. ‘Hey, I know you asked for this, but I thought about doing this. What do you think?’”

All told, Herbst concludes given these challenges, everyone was satisfied with how Smallfoot turned out. “Working with these filmmakers, we had a great relationship, and they trusted us. We wanted to provide everything they had on their wish list, and I think in the end everybody was really happy with the film. So, even though it was a rough ride in terms of changes, starts, stops, redoes, things like that, it ended up working out. We all had a great time.”