A Behind-the-Scenes Look at the Heart and ‘Soul’ of Pixar’s Latest Feature

“Soul,” which opened on Disney+ on 25 December 2020, tells the story of Joe Gardner, a middle school music teacher and aspiring jazz performer whose soul separates from his body after an untimely accident. The film, which is Pixar’s first Black-led feature, was the focus of several SIGGRAPH 2020 Talks presentations and was completed from a distance during the COVID-19 pandemic. We caught up with Pixar’s Michael Fong (supervising technical director) and Bill Watral (effects supervisor) to hear more about how they created the characters of “Soul,” the contrast between the settings of New York City and the astral world of the Great Beyond, the most impactful scenes, and what it was like to wrap a film while working remotely during the pandemic.

SIGGRAPH: The characters in “Soul” are more abstract than other Pixar film characters. What techniques did you use to ensure the characters would appear “non-physical” and “formless”? How did this differ from other films you’ve worked on?

Michael Fong (MF): The characters are really different for us. We were in the art department looking at various art designs and Deanna Marsigliese came in with these designs for the counselors that made our jaws drop. She had made these little forms of wire, which were totally different from the kind of models I’d done previously. I remember coming out of that art review, and one of the supervisors turned to me and said, “You know, that design really scares me. That’s how I know it’s going be awesome.”

And it actually really scared me to death, because not only was it going to be made out of wire which is in itself a problem for us. Her idea was that the body parts were constantly changing. There could be arms on the left side and then all sudden, they would transition over to the right side, but not by any means of physically moving the arms over. The arms would actually sprout from the other side.

She also did this unusual thing where she had a flashlight, and she was showing what the characters look like when projected on the back wall. She had set up a little diorama. She said, “I really like how you can’t tell the forms anymore in 3D when you see the projection on the back wall. It’s really strange all the faces get distorted all the fingers to distort it.”

Pete [Docter] really responded to that. His whole point was that this world was full of characters who were trying to look normal, but they weren’t quite succeeding. He really wanted characters that didn’t look like your normal Pixar flares. We worked hard trying to figure how to get rigs to work with moving body parts and how their eyes would float in the air.

We also had to figure out how to shade the characters, making them look like they were made of particles from the ground. You can have shapes that don’t allow you to make membranes, because they’re not actually in the same plane. It involved a lot of tricks and hacks to make it all feel like these were both 2D and 3D at the same time.

Bill Watral (BW): For the volumetric characters, it started in art with little puffs of cotton. We took a filtering class to explore that, as well.

There was this challenge that we had to create these characters that felt like they were these cotton balls or these opalescent kind of things, and creating volumes was not a pipeline that we actually had in place in the characters team on the front end. There were technical complications involved in that. Early on, the character group rolled up their sleeves and decided to figure out what tool set is best to do this. They settled on the authoring environment that we use in effects to generate volumes and figure out how to build the rigs in a traditional Pixar way in our proprietary system, but then process all that data through the software that we use to generate volumes and effects. There was a bit of a handshake up front to try to figure out how to use the effects tools to process what animation was producing in the performances of the characters. Then, to the characters department’s credit, they took off with this tool and did all this crazy stuff that we in effects had never even thought of the characters that way. They used that Houdini tool set to do a lot of that trickery and hackery, as well.

It was really fun upfront to do this partnership with the character team and teach them some of the effects tools. They took off with it and made these amazingly complicated, under-the-hood characters, but when you look at them, they’re these little, simple expressive pops on the screen that are really emotive and expressive, and it belies the fact that there’s so much technical stuff going on under the hood to make those characters look like what they look like. It’s pretty cool.

© Disney/Pixar

SIGGRAPH: Tell me about the most challenging technical aspect of creating “Soul.” Can you share an example of a problem you faced and how you overcame it?

MF: The big thing for us was trying to find the design or look of the astral world in “Soul.” You can always visit a place like New York to reference it, but the astral world was something that existed in Pete’s mind, and we had to tease it out. In the beginning, we would get all these pictures that were very impressionistic, and I had no idea what we were looking at. I would be pointing at things and saying, “What’s that? Is that part of the character? What are these things?”

One of the biggest challenges for us was actually discovering what the look was for the astral world. Normally, we try to find these designs for our sets inside the art department, but in this case we had to share the responsibility. The art department said we can only go so far with 2D, and we said we’ll try to do 3D, but we can only go so fast. We tried to put these two teams together to figure out what these different worlds were.

When you see the movie, there’ll be several different astral worlds. There’ll be a place where the souls are being given personalities, but there are other places like the astral sands area. There’s the “Hall of Everything,” where everything you can possibly imagine is and you examine all these different worlds and different looks.

BW: I feel like that was really the most technically challenging thing on the film, these two completely different worlds. The one was easy to figure out. The other was insanely challenging, and the amount of resources and brain power that went into generating things that never ended up on the screen, to find those looks in that soul, is mind boggling. There is so much cool stuff that the world will never see, or at least they’ll see a very small sliver of it in the film.

There were people from effects and simulation and art and lighting all working in this group that was doing really fast looks in the development cycle to just throw as many pictures up on the wall as possible and see how people responded to them. From a technical standpoint, it was pretty daunting figuring out how to implement that … even when we had the right artistic direction.

SIGGRAPH: “Soul” features scenes from both New York City and the astral world, The Great Beyond. How did you create these two distinct worlds, and what do their contrasts add to the story?

MF: [Steve] Pilcher and Pete always talk to us about a contrast between the two worlds. I think the words that Pilcher always used were “ephemeral” versus very “hyper-textual” and “tactile.” They really wanted the astral world to be a place of softness, of welcoming and warmth for little kids, almost like a nursery in some way, like you can almost tell what all the pieces are but the one material ends and another one starts.

Whereas when they went to New York, they wanted it to have a lot of texture. There were rugs that would have lots of fibers so you could imagine feeling them or feeling the brick of a wall. It was very, very tactile.

BW: I feel like the juxtaposition of the two worlds is this rigid surface that we all recognize as a city and a world that is real. We wanted the audience to recognize the other construct as a place that could be inhabited, yet also have a feeling that it was dynamic and a representation of something bigger, underlying the way the universe works and was defined enough that it felt like a space, but soft and confusing enough that it felt like you were questioning it along the way the whole time. Putting those two worlds together and against each other, particularly when the scenes cut from New York to the astral, is really interesting and intellectually stimulating for me. Even when I watched the film and I know all the underpinnings that went into jumping back and forth between the two worlds, it’s really pleasing to have that visual juxtaposition.

© Disney/Pixar

SIGGRAPH: What scene from “Soul” are you most proud of?

BW: There are so many, you know, I love all my children. I’m most proud that we did manage to pull off the “Soul” world in a way that when I watch it, I believe it.

I don’t pick it apart too much. If I worked on a shot, I do, but if it’s the other stuff that went on around, I buy it. I believe it and it feels like a place that really pulls me in. This was such a huge group effort. Everybody who worked on the film contributed to it coming to fruition.

Anytime you see something out there on what we call the “astral plane,” those are the shots that resonate with me, and I feel proud of the whole team for managing to pull together and make something that is convincing and believable.

MF: The scene I’m most proud of is not because of something technical. There is a time when Joe visits his mother. His mother works at tailoring shop, and I really like it because it’s a scene that speaks a lot about Joe and his relationship with his mom. There also was a lot of research put into it, what a tailor shop might look like, what it would look like in Joe’s community. We did a lot of research visiting those places, and the artists came back so proud, noting details like the pin cushions tailors wear on their wrists and the way the mannequins look in the shop. The artists were really excited about these scenes. I think that love and excitement shows in the scene, even though it seems like a relatively quiet scene when Joe’s talking to his mom. It’s a really great scene where people have thought a lot about where the mirrors are placed, how the clothes are being hemmed, how the sewing machines and lights are set up. I just love the detail on that scene.

SIGGRAPH: In “Soul,” Joe Gardner is passionate about his music. What are you most passionate about in your career? Where do you find inspiration?

MF: The thing that inspires me most, what makes me passionate about computer graphics, is not necessarily always the computer graphics, to be honest. It’s when I’m watching teams like Bill’s team or the character team or the sets team take these problems that Pixar hasn’t dealt with before and figure out how to solve them. My first gut reaction upon seeing designs coming out of artists is, “How are we going to afford to do this? How are we going to get this done in time? You only have two years to make this movie.”

And then Bill comes in. He says, “We got this. I have some ideas. I’m going to be working with my team, and it’s really great.”

From my point of view, I’m inspired by seeing these teams solve really hard technical problems.

Sometimes, they have to “cheat.” I’m not saying that we’re doing the most pristine ivory tower sort of research. But the end product is something beautiful, and they inspire me a whole bunch.

BW: I think there’s so much amazing moving picture work done out there in the world, and every time I look at something, I feel like there’s so much room to do better. From an inspiration standpoint, I like seeing all the amazing work that’s going on. Nowadays, technology is a lot more accessible than it was when I came up in the industry. There are machines and free software that do amazing work. Keeping myself exposed to all that work out there in the world is what inspires me. There are students making things that are as good, or maybe even better, than the stuff that we’re doing. It’s really, really inspiring. It’s really cool.

Then, I juxtapose that with the world we’re all inhabiting. I find a lot of inspiration from being outside in nature and taking walks, looking at the world around me and trying to find those little details like Michael talked about with the sewing shop. That’s the other thing that inspires me —  I’m on a hike and I swish my foot in the water, and there’s all these little bioluminescent creatures that appear. It’s a combination of those two things, really, that kind of keeps me moving forward.

SIGGRAPH: What do you hope viewers take away from “Soul”?

MF: Pete is always telling us that we all have big aspirations in life, but there are small things that are going on all around us. You have to make sure to enjoy the big things and the small things. The small things make up life, and if you don’t pay attention to it, your life passes by and you don’t realize it. I think one of the messages the film shares is that there are great things everywhere, and you have to enjoy it.

BW: Hearing you say that out loud got me a little emotional. We finished this film as the pandemic unfolded. At the start of the pandemic, we all grabbed our remote boxes and our monitors, put them on our office chairs, and rolled them out of the studio into our cars and went back to our houses to figure out how to finish this film. In doing so, I feel like I found some of the message that Pete hoped.

My commute became walking from my back door to my garage door where I set up my machine. I really appreciated hearing a bird singing as I walked from one door to the other to go finish “Soul.” That was a different experience, and I think the timing of the release is really proving it, as well.

© Disney/Pixar

SIGGRAPH: This year has had many of us transition to a remote work environment. Can you share a bit more about your experience finishing this film from a distance and how that impacted your creative endeavors?

BW: We were in the thick of it in effects when the pandemic hit. We had a whole series of shots that you’ll see toward the end of the film that were very effects-intensive, and we were in the middle of getting them done. We had one shot in particular that was still on the drawing board as to how we were going to solve it. The technical details of it were still in process.

Trying to get everything set up and communicating and reviewing the work in a creative way was the biggest challenge. It was so awkward to transition over to this communication style where Zoom cuts off your voice when somebody else talks, and there’s three different pieces of technology that you need to use to view an image, and only one of them works at the frame rate that you need to see it at, but doesn’t work for color correct or other technical details of things. It was so amazingly challenging.

My hats are off to our systems people at Pixar for cobbling together whatever was there technology-wise to make stuff work for us to get it done. They went above and beyond. It was amazing how much progress we made in two or three weeks from when we first plugged in our little remote boxes to when we started getting the shots finished for the final film. And then, there was all this fear of what this stuff looks like up on the big screen versus our little screens in our home offices.

Source:Siggraph