Atop an elevated train barreling through Downtown, the masked man in the red and blue suit is in trouble.
Not only is he fighting a lunatic scientist who's trying to kill him with robotic tentacles, but he also needs to save the passengers on the train. It's all in a day's work for superhero Peter Parker, also known as Spider-Man -- but it means months of work for an elite team of graphics gurus.
This action sequence from the megahit Spider-Man 2 has dazzled millions of moviegoers this summer. Look closely at the villain, Doctor Octopus (played by actor Alfred Molina), and you'll see him grin maniacally and yell while ambulating along the top and sides of the train. Later in the scene, Spider-Man (Tobey Maguire) loses his mask as he braces himself against the front of the train, bringing it to a screeching halt. His frantic facial expressions appear convincingly natural.
With all the swift cross-cutting between images of superhero and villain, the audience probably does not suspect that the faces and figures appearing on the screen much of the time are not the real thing. Rather, they are digital concoctions created inside a computer at Sony Pictures Imageworks in Culver City, CA.
"We've reached a point where we can make every single thing computer-generated. Everything from the shattered glass in the train to all the buildings, tracks, and people," says Mark Sagar, a graphics supervisor on Spider-Man 2 and Imageworks' resident expert on digital human faces. He points to the train sequence as it plays on his computer screen. "Watch what the camera does. Here it's going fast along the train, then underneath the performers, then in front of them, then a wide shot. How could you do that with a real camera?" But the digital faces on the people are the last, crucial piece of this puzzle. In the past, splicing footage of real actors into a digital scene required real cameras, difficult stunt work, and tweaking to get the look of the real and digital images to match; the ability to do computer-generated everything, including human faces, opens a wealth of creative possibilities.
Photorealistic digital faces -- ones that can pass for real in still pictures or on the big screen -- are among the last frontiers of computer graphics. Until recently, digital faces have looked fake when examined closely and have therefore been relegated to quick cuts and background shots. The problem is that we're extraordinarily sensitive to how human faces should look; it's much easier to fool people with a computer-generated T. rex than with a digital human. But advances in rendering skin, lighting digital scenes, and analyzing footage of real actors for reference are now allowing artists and programmers to control the texture and movement of every tiny patch of pixels in a computerized face. The Sony team hopes that audiences will be unable to tell the difference between Tobey Maguire and his digital double -- perhaps the first time such verisimilitude has been achieved.
The stakes are huge. Digital effects are a billion-dollar business and growing fast; these days, a typical blockbuster film's budget can be $150 million, with half of that going to effects companies. Indeed, Spider-Man 2 is just one example of Hollywood's increasing use of cutting-edge graphics research to create better digital actors, from stunt doubles of Neo and the multitude of Agent Smiths in the Matrix films to Gollum in the Lord of the Rings series. It's reached the point where the industry jokes about replacing actors with computers (the premise of the 2002 film S1m0ne). And Sony Pictures Imageworks, founded in 1992 and with more than 40 feature films to its credit, is in the vanguard of effects houses that vie for big studios' business .
But the real benefit of digital actors isn't replacing live ones: it's delivering scenes that take viewers to places no real actor, or camera setup, could go. "It's giving directors more flexibility, and it allows them to do actions they can't do with real stunt people," says Scott Stokdyk, the visual-effects supervisor at Imageworks in charge of the Spider-Man series. "In the past, directors and editors have basically massaged the cut around different quick actions and camera angles to convey a story," Stokdyk says. "Now they don't have those kinds of limits." Thus liberated, directors can follow synthetic actors as they swoop around skyscrapers and dodge bullets in sweeping slow motion. What's more, actors can be digitally aged, or de-aged, without having to spend hours in makeup. Long-deceased movie stars could even be digitally resurrected.
And movies are just the beginning. Techniques for creating digital humans are pushing the broader frontier of computer graphics and interfaces. These efforts could enable strikingly realistic medical training simulations, lifelike avatars for e-mail and Internet chat rooms, and soon, much more compelling characters in games and interactive films. The technology, Sagar says, "is absolutely ready for prime time."