Disney Is Getting Into the Deepfake Game
Scientists affiliated with Disney Research Studios have come up with a way to use deepfakes to swap out faces in high resolution for the first time.
That could mean endless possibilities for future live-action films.
The scientists are presenting their paper at this week's Eurographics Symposium on Rendering 2020.
There's a whole trove of blogs and YouTube videos devoted to face swaps of some of the most beloved Disney characters, from Ariel with Ursula's face to Lilo and her doll Scrump.
But these are just cartoons. What if the Disney could seamlessly—and convincingly—switch out the faces of real actors in live-action films like the Pirates of the Carribbean franchise?
New technology from Disney Research Studios could make it happen on the big screen for the first time. Four scientists have come up with the world's highest-resolution deepfakes, or synthetic media in which a computer program can use existing images to manipulate someone's likeness.
"To the best of our knowledge, this is the first method capable of rendering photo-realistic and temporally coherent results at megapixel resolution," the researchers say in a paper they'll present at this week's Eurographics Symposium on Rendering 2020.
Sure, megapixel-level resolution doesn't sound groundbreaking, considering the resolution of some smartphones, like the Galaxy S20 Ultra, snaps photos at the megapixel scale in the three-digit range. Still, it's a humble step up from current state-of-the-art deepfake tech.
The researchers note that DeepFaceLab, a popular open-source deepfake system, could only create face swaps at 256 x 256 pixels, while their model could produce 1,024 x 1,024 pixel resolution.
To create the deepfakes, the researchers begin with a recording of a person exhibiting some desired behavior, like talking. Next, they perform face alignment by detecting facial landmarks, or certain shapes that correspond to given facial features—the same underlying principle that facial recognition software platforms use.
Then, the researchers "transform and normalize" the image to 1,024 x 1,024 resolution and feed the image into their decoder program, which swaps the input face with a target face.
The ability to switch out the appearance of one face for another could be endlessly useful for a massive media company like Disney. Usually, it takes elaborate and labor-intensive visual effects (VFX) methods to make an actor appear younger, or to use footage of a deceased actor in a film that hasn't yet wrapped filming. Consider Paul Walker in Furious 7, Carrie Fisher in Rogue One, or the forthcoming movie starring the long-dead James Dean.
Traditional VFX methods "require great care on set as well as extensive frame-by-frame animation and post-processing by digital-effects professionals," the authors say. But they believe that, with continual improvement, their deepfake technology could cut down on the time and cost to create these convincing doubles.
Now all we have to do is hope this powerful technology doesn't get into the wrong hands.
You Might Also Like
