Is Tom Hanks multi-lingual? No, it's just AI

Is Tom Hanks multi-lingual?

No, he’s just getting a little help from AI.

Startup Flawless AI has been working on making dubbing in films less… distracting.

The company studies in intricate detail all the mouth movements an actor makes, and swaps them out according to the dubbed words in different languages.

The end result makes Jack Nicholson look like he's fluent in French.

Scott Mann is a film director and co-founder of Flawless AI.

"Look, personally, I hate dubbing, because you have to change so many things to try and catch sync so you're changing words, words that filmmakers and performers we've kind of all thought about so deeply but they're just kind of thrown out to find a different word that kind of fits sync but it never really does. Like whenever we watch a dub, we always know it's not in sync and what that leads to is a non-immersive experience and so it pulls you away from feeling films and feeling content. And that's damaging and it really lessens the experience and it puts people off wanting to do it, you know."

The idea began when Mann saw how dubbing affected the narrative cohesion of his 2015 film "Heist", starring Robert De Niro.

After some research, he discovered a white paper by Christian Theobalt of the Max Planck Institute for Informatics, laying out a new approach that uses artificial intelligence to recreate photo-real human visual effects.

"Essentially it's studying in intricate detail all the nuance of human pronunciation and all the visemes that are made during mouth movement and it's able to essentially take, say, an 'ooh' sound from De Niro 20 minutes earlier and place that in a different moment in the film and it kind of measures at the same and blends it so that the performance is the same but it's a different mouth movement."

Flawless AI is now working with producers and studios to integrate the technology into post-production.

Mann believes that in time, the tool will help eradicate foreign language categories at award shows, and create a more inclusive film industry with a real global stage.

He said the first films using this tool are around a year away.

"I know that the first few things that come out, people will probably be looking out for it in particular and will be studying it. We'll all do it, I'm sure and it's interesting like that but look. We're all about doing this at scale for everyone and I think when it's a common thing we're not going to think about it anymore and we'll just enjoy the movies."