Translation with AI in Cinema: The End of Voice Actors?

Artificial Intelligence (AI) is revolutionizing the cultural landscape in an unimaginable way. One of the most recent innovations is technology that not only translates words but also synchronizes lip movements to match the desired language. This raises fascinating questions, particularly in the realm of cinema.

First, let’s explore how this technology could impact the film industry. On one hand, it could mark a turning point for voice actors and subtitling companies. If AI can automatically translate and sync an actor’s lip movements, the need for voice actors could significantly diminish. Similarly, subtitles might become less necessary if AI can provide a more immersive viewing experience.

A recent video published by @mrjonfinger on Platform X went viral, demonstrating the capabilities of early versions of translation software. In the test, as described by the user, he recorded a video speaking for over 30 seconds. The platform then generated two versions of the text: one in French and another in German. Remarkably, his lip movements synchronized with each language, adapting gestures to match the sounds produced. Despite being one of the early versions of the software, it exhibited impressive consistency and credibility in both tone of voice and facial movements. If this is the first version of the software, determining the original language in future iterations may become increasingly challenging. You can watch the video on the link below:

https://x.com/mrjonfinger/status/1701075571630047525?s=20

How can industries offering these services adapt? Instead of resisting change, they can evolve into specialized AI translation services, becoming the direct link to using this technology in film editing. By working closely with editors during post-production and overseeing the process from start to finish, they can offer a wide range of languages for global distribution. Their primary role would involve closely observing actors’ facial movements and correcting any errors that may arise, as, let’s admit it, machines can make mistakes. This approach would be analogous to the work of subtitle inspectors since the advent of automatic subtitling technology, ensuring that all words are correctly placed and convey the appropriate cultural nuances, especially in…