At UX Week 2008, our Day 4 keynoter is Dr. Michael B. Johnson, who runs the Moving Pictures Group at Pixar. He’s been gracious enough to engage in an email conversation with me, which I’ll be sharing here. For more Michael, register for UX Week. Use the promotional code BLOG and get 10% off!

Peter Merholz: UX Week 2008 is devoted to the discussion of experience design, and we’re excited to have you on the program because Pixar is the premier studio for delivering consistently high-quality cinematic experiences. When it comes to animated filmmaking, the folks reading our blog will be familiar with the roles of the writer, director, editor, animators, voice actors, music and sound effects, those things that are apparent when sitting in the theater and staring at the screen. I suspect, though, that if they even know that such a thing as a “pre-production pipeline” exists, they have no idea its function in the filmmaking process. So I was thinking it’d be best to start there. Given that WALL•E has recently opened, it might be helpful to use it as a point of reference. When did you begin working on the WALL•E production? How did you and your team support it?

Michael B. Johnson: Historically, you can look at the Pixar film making process as one of building a prototype version of each film (known as “story reels”) and then moving on to building the fully realized one. This process has continued to evolve over time, where (perhaps surprisingly) the polish of the story reels has been rising even faster than that of our final films.

The Incredibles really set a new bar for the effectiveness of story reels. I think that had to do with particular leadership in place on that film - director Brad Bird, head of story Mark Andrews and director of photography Andy Jimenez. They’re all super talented, and on The Incredibles their skills perfectly complemented each other. The reels for The Incredibles are jaw-dropping. I think they’re good enough by themselves to be a released film. Rough, 2D, black and white - and completely compelling. They really show what story reels can accomplish - getting a 3D crew to see them and just think to themselves “now I just need to not screw that up” :-).

In the midst of Incredibles (2002), I was part of a team that put together a review/sketching system for Brad Bird, discussed in this article.

I also started working on a digital storyboarding tool for Pete Docter, who had just finished Monsters Inc. I originally called the tool “Pete Docter’s Tool”, a nod to Pixar’s original animation system “Motion Doctor Tool”, but then Angus MacLane suggested “Pitch Docter”, which is what we went with.

Before The Incredibles, storyboards were drawn on paper and then delivered to Editorial, where they were scanned and brought into the computer. On the Incredibles, scanning moved back to Story, where Mark Andrews (the Head of Story) would do a pass in Photoshop to make the images have a consistent “look”.

I showed what I was doing with Pitch Docter to Brad Lewis and Jan Pinkava (producer and co-director of Ratatouille, respectively) and asked if they were interested in trying this for storyboarding on their film. They said they were, and that really started the current incarnation of a “pre-production pipeline” here that my group has been working on since 2002.

On Ratatouille, for the first time we had many of the story artists working full-time in Photoshop, leveraging its brushes, layers, and actions to streamline their workflow. They used Photoshop in conjunction with Pitch Docter, which let them time out their pitches, add sound and dialog, and round trip with Editorial. I’ll talk a lot more about this is in my talk.

The other department that’s vital to the early development of the film is the Art Department. These are the folks who design the look of the film — the characters, the world.

The issue with these folks is not so much their internal workflow, as much as the way they share their work with other departments - when, what, and how. Again, this is something I’ll speak to in my talk.

The important take-home point, though, is that Pixar loves their films so much, we make them twice :-). Compared to the final product, the first time we make it is sketchy and rough - but the most important thing is that it’s still a film. To be clear - our prototype exists in the same medium as our final product. This allows us to judge it by the same standards that the final film will be judged.

I think this is an important lesson for a User Experience Designer to understand - paper prototypes and ethnographic research are great, but if you’re trying to build a prototype that you want use as a blueprint, it should exist in the same medium as the final product. My group (which does lots of ethnographic research and Photoshop/OmniGraffle prototypes) firmly believes in this, and practices it daily.

With WALL•E, my group got involved in early June of 2004. We were in the midst of working on Ratatouille when WALL•E started gearing up. Originally, we started helping their Art Department understand their technological choices available, and then pretty quickly we started helping them understand what was involved if they wanted to storyboard their film digitally.

In talking to Andrew Stanton, we quickly realized that WALL•E would be pushing our process much harder than any earlier Pixar film. Part of this was his approach; with less character dialog, there was a need for more images to make the story explicable. Also, Andrew was reasonably comfortable with technology; as long as our tools could keep up with his thought processes, he was interested in using digital sketching to do his reviews (our efficacy at this waxed and waned over the course of the production, which was itself very educational with respect to the “user experience”).

In short, WALL•E was a great proving ground for a lot of our technology, and was a wonderful cauldron to prove (and disprove) a wide range of approaches we took to bringing digital technology not only to the artists’ desks, but also to the director, where the tolerances are much tighter.

Anyway, I’ll have lots of great stories to share about this at the conference.

PM: Fascinating background. I love that you produce a completed (though sketchy in appearance) film as a prototype. At Adaptive Path, we’ve been moving towards this approach ourselves. As the experiences we design get necessarily more complex, it’s not sufficient to design them as static sketches, and hand them off to others to implement. We’ve been creating richer and richer prototypes, not as an end-product of design, but as a step in design exploration. If you’re going to design for experience, you’ve got to understand what it means to physically engage with your design as quickly as possible, and be prepared to change those designs as need be.

Your commentary raises two distinct questions for me.

First: To make the entire movie twice seems like a significant cost. How has it been deemed worth it?

MBJ: Actually, it more than makes up for the cost. We know we’ll fail a lot; if you don’t fail you’re not doing anything new :-).

We’d much rather fail with a bunch of sketches that we did (relatively) quickly and cheaply, than once we’ve modeled, rigged, shaded, animated, and lit the film. “Fail fast,” that’s the mantra. With a team of 10-20 people (director, story artists, editorial staff, production designer and artists, and skeleton production management) you can make, remake, and remake again a movie that once it hits 3D will take an order of magnitude more people to execute. The complexity of the task does not ramp up linearly.

PM: Second: There seem to be an awful lot of people and roles to coordinate. In our work, one of the biggest challenges we see facing organizations is how to coordinate the efforts of cross-functional teams, often comprised of people working in distinct organizational silos. Most organizations approach this by engaging in some form of the waterfall approach, where product development is handed off from silo to silo in the organization until completed. I get the distinct sense that Pixar’s approach is a lot more “all hands on deck.” How do you coordinate the efforts of so many distinct contributors?

MBJ: I would by lying if I said we knew what we’re doing :-). I think we’re starting to get the hang of it after 9 feature films, but it’s hard. Production management is a hard, hard problem. Like all things at Pixar, casting the right people in the right roles is the most important starting point, but we’re constantly refining/reinventing our processes to work for the problems we have with the people we’ve got.

I always stand in awe of good production management (which we are blessed to have). They keep a lot in their heads, and they juggle a ton of data within a complex web of constraints. Part of my job is to make sure that we track the right things, and make that data transparent to them, so they can generate the decision making information they need. A film is a big pipeline, and there are hand offs between departments, but there’s a lot of iteration and back channels. A lot of it is getting the right people talking to each other, removing barriers to communication.

One of my heuristics for thinking about how we (the designers and technologists) can help with production management is to look at where people are getting mad each other. This usually indicates some frustrating breakdown in the information flow. When people are getting bad/late/incomplete/stale information, they get frustrated. These projects take a long time to make, and like any business, there are always going to be areas where communication breaks down. When that happens, our team works on fixing the information flow.

Morale is super important; assuming a competent team, it’s probably the most important thing for a long project. Brad Bird has a great quote in the interview he did with McKinsey a few months ago:

“In my experience, the thing that has the most significant impact on a movie’s budget–but never shows up in a budget–is morale. If you have low morale, for every $1 you spend, you get about 25 cents of value. If you have high morale, for every $1 you spend, you get about $3 of value. Companies should pay much more attention to morale.”

PM: Brad’s comment about morale is revealing. Pixar has achieved remarkable success — with the release of WALL•E, it looks like you’ll be 9-for-9 in terms of hit films, all of which have received favorable criticism as well.

No other studio even comes close. And not for a lack of trying — there are many truly gifted people working in film. The challenge, I think, is that on the production of a major motion picture, there are so many parts, from concept to casting to production to marketing, and then so many things out of your control (weather, economy, current events, the societal Zeitgeist) that it’s seemingly inevitable that when you mix these elements together, sometimes you’ll get a dud. The exact same people can make two different films, and one will be great, and the other not so much, and it’s because of all these other elements.

You’ve been at Pixar for 13 years, and I believe your time there has exposed you to pretty much every aspect of film production. From that insider’s perspective, what can you impart about how Pixar is able to be so continually successful? What have you figured out that others have not?

MBJ: Fundamentally, people at Pixar respect each other, and in most cases, even like each other :-). We are making movies and shorts that we want to see. We’re not afraid to take chances, and we know we’ll fail along the way, but we do a good job of making each failure part of the process and use it to get to something that we’re happy with.

As Edna Mode says, “Luck favors the prepared, darling”.

As you say, we’ve been at this for a while, but we are under no illusions that we know what we’re doing. We do have some real experience under our belt, but I don’t think anyone here would tell you they’re done learning/growing/challenging the processes we use. A lot of the leadership of the films (directors, production designers, creative and technical leads, production management) have been working together in different configurations on films here for over a decade. We have animators, our actors, that have animated on almost every film we’ve made. I like the line: “The harder I practice, the luckier I get.”

I also like a comment I’ve heard Andrew Stanton say, which is “talent isn’t fair.” I’m lucky enough to work at a company where I don’t have a chance of being the smartest person in the room, and I like it that way. I won’t lie; it’s hard to work with so many talented people, you have to have a certain diamond hard sense of self or you can come home bummed out after a hard day at work. But it does cause you to bring your A game. Luckily, we tend to do a very good job of hiring people that are actually nice, and really want to work with other people.

I think it speaks to the fact that you need to assemble the right team of talented people, and inspire them to work on something great, and they will. It almost certainly won’t be the thing they thought they were going to make, but as long they keep true to the high level vision of making something that appeals to them, they’ll be successful. I think it helps that our creative leads here have sensibilities that resonate with the audience at large, and I honestly don’t know how much of that is about the earnest and truthfulness of the execution and how much of it is the subject matter. But at this point we’ve been successful with movies about rats in kitchens and trash compactors on a dead planet, so I have to think it’s the love of the story showing through and catching the audience.

Share This

Of all the ideas we talked about while designing Aurora, we kept coming back to a core set we considered essential, high-priority elements of the browser. These ideas clustered around four major themes:

Context awareness: Is there another product that has the potential to know as much about us as the web browser? Not only does the browser touch every aspect of our lives — our work, families, social connections, entertainment — but the data that flows through it is so semantically rich. If the browser paid attention to all that data, and also paid attention to our behavior as we interact with that data, it could find patterns and adapt itself to ease the difficulty of managing our interactions with the Web. Add to that the ability of the browser to be aware of your physical context, and the possibilities expand even further.

Natural interaction: Most of our interaction with technology involves levels of abstraction. Windows, menus, and toolbars are notional objects bearing little resemblance to real ones. The trouble is that dealing with all these abstractions is hard work, cognitively. The brain really wants to interact with a familiar system: the real world. So we designed Aurora to leverage natural interactions wherever possible, with objects in space or those with a sense of physics to them. The Mozilla team liked this approach as well: one of the core Aurora concepts, the spatial view, has already found its way into some of the work Mozilla is doing for Firefox Mobile.

Continuity: Another area that both Mozilla and Adaptive Path were keenly interested in exploring was the idea of continuity of the browser experience. We didn’t want to design different interfaces for desktop, handheld, and wall-mounted devices. We wanted to come up with a single, consistent interaction model that could apply no matter what size screen you were using, or what means of interacting with the device (mouse, touchscreen, gestural controller) you had at your disposal. Also, we wanted to explore how the Web experience could be seamless across devices — so people could move from one context to another, always picking up right where they left off.

Multi-user applications: The Web is something that people use together. But the browser has historically been a single-user application. A browser built with multi-user applications in mind could provide a platform for much of the functionality we now see being re-implemented and reinvented on a site-by-site basis. Collaborating simultaneously in a common space, sharing information with others, and recombining or remixing elements from the Web all become common, assumed functions of any website.

Creating Aurora sometimes challenged us in ways we didn’t expect. In a typical design process, one of the biggest factors influencing the design is the set of constraints we have to work within — not just the limitations, but also the criteria for success for our work.

A good designer can create a design that accommodates all the constraints and still delivers an elegant, satisfying experience to the user. A great designer can go beyond this and create a design that demonstrates that some of those constraints weren’t really there to begin with. But when you’re designing for the future, all of your constraints are imaginary. Making smart choices about the constraints you create for yourself makes the difference between a plausible solution and science fiction.

But with a problem like designing the browser of the future, we weren’t even sure where to start. The evolution of the browser seemed to be intimately intertwined with the evolution of the Web — and to some extent, the underlying Internet — itself. Plus we had to account for trends in general computing technologies: smaller, faster, powerful, more connected and ubiquitous devices, enabling new kinds of interactions and applications.

To help us get a handle on all the possibilities, we asked Jamais Cascio to contribute some time to the project. Jamais is a professional futurist who forecasts trends for organizations that will drive their strategies on timelines quite a bit longer than the next quarterly earnings report. He co-founded the popular blog Worldchanging and runs his own blog called Open the Future.

Jamais called on a whole lot of smart people and led them (and a bunch more from both Adaptive Path and Mozilla) through a two-day workshop to forecast one possible future for browsers and the Web. Through a series of group exercises, we identified three major trends that we thought would have the biggest impact on the web:

• Augmented Reality: The gap is closing between the Web and the world. Services that know where you are and adapt accordingly will become commonplace. The web becomes fully integrated into every physical environment.
• Data Abundance: There’s more data available to us all the time — both the data we produce intentionally and the data we throw off as a by-product of other activities. The web will play a key role in how people access, manage, and make sense of all that data.
• Virtual Identity: People are increasingly expected to have a digital presence as well as a physical one. We inhabit spaces online, but we also create them through our personal expression and participation in the digital realm.

Based on these trends, Jamais wrote three scenarios fleshing out the details of how these trends might come into being, and how they would manifest in people’s everyday lives. We wanted to use these forecasting scenarios to explore several aspects of this possible future world that we knew would never end up in our movie, but would provide us with some context for the design choices we’d be making.

Download:
Forecasting Scenarios

Forecasting Workshop Contributors:
Mike Beltzner
Rebecca Blood
Stowe Boyd
Leah Buley
Dawn Danby
Alex Faaborg
Henning Fischer
Jesse James Garrett
Dan Harrelson
Sebastian Heycke
Julia Houck-Whitaker
Mike Liebhold
Jessica Margolin
Peter Merholz
Lisa Rein

Tomorrow on the Adaptive Path blog: The technology of the future!

Early in the process of planning the Aurora project, it became apparent to us that to showcase the browser of the future, we’d have to design the Web of the future. We decided this was a great opportunity to ask some of the smartest, most creative people doing Web design today to volunteer their time and contribute their perspective on where the Web is headed. We want to thank all of the designers for their contributions to the project:

Dave Shea was the first designer to come in on the project, and demonstrated extraordinary patience with our shifting requirements as we were finalizing the script. You can see his work throughout Parts 1 through 3.

Andy Rutledge and Angela Conlon of Unit Interactive contributed some innovative thinking and great visual style in their vision of the future of the New York Times in Part 1.

Chris Glass tackled some of the most complex design work for Part 4. He also contributed a finished design for a sequence that was cut from the video late in production.

Our own Alexa Andrzejewski stepped in on short notice to deliver great designs for the personal weather station and the rainfall line graph in Part 1.

I’ve written before about Adaptive Path’s open design sessions: regular open-invitation forums that allow project team members to draw on the perspectives of everyone at the company, and allow everyone to learn from projects other than their own. In a typical project, the team might have the opportunity to hold two or three open design sessions. For the Aurora project, we knew we’d need a lot more input than that. So we instituted weekly open design sessions, carrying us through from the earliest feature brainstorming to developing ideas for the visual design.

These sessions were an invaluable vehicle for collaboration between Mozilla Labs and Adaptive Path. Every Friday afternoon, we’d spend a couple of hours with a rotating group of contributors from Mozilla, talking through the design ideas we’d been coming up with, and seeing how those aligned with Mozilla’s own thinking about the future of the browser.

Early in the project, when we were still trying to give shape to the features and functionality of the future browser, these sessions were more freeform and speculative, and to an outside observer their connection to the ultimate design work may not have been obvious. But for us, this process was an invaluable tool in defining for ourselves the landscape of possibilities.

In between the open design sessions, Dan Harrelson and I would incorporate the input we’d received into our almost-daily collaborative work sessions. Usually, each of us would have some facet of the problem that had been nagging at us since the last time we met, and we’d take turns talking through those issues, sketching endless possibilities on whiteboards.

Much of this process involved questioning our own assumptions about what would and wouldn’t work. Although human behavior doesn’t change at a pace anywhere near that of technology, we did think that we could expect people to adapt some of their behaviors (and maybe adopt some new ones) as technology continued to evolve. The big question was which behaviors were long-term ones, and which were ripe for (relatively) short-term change. For every idea that made it into the final design, we probably discarded a dozen alternate approaches.

We found many of the design concepts difficult to document. Our usual tools like wireframes and mock-ups just seemed unsuited to an interface as dynamic as Aurora. Because so many of the conventions of interface design didn’t apply to Aurora, I developed a “design concepts document” that described the basic principles of how users would interact with Aurora and how elements of the interface would behave, so that Kumi Akiyoshi and Sebastian Heycke, who were responsible for the visual design of Aurora, could be sure they understood what we were trying to accomplish. This document evolved into a considerably simplified “interface guide” that summarized the various interface elements and how they work together.

For more detail on the Aurora design:
Aurora Design Concepts
Aurora Interface Guide

This video is best viewed in high definition. To view at 1280