Voice actors are training the AI that will replace them

Earlier this year, Alejandro Graue got a YouTube alert: A new video had been uploaded to a self-improvement channel for which he’d recorded voice-overs in the past. But it wasn’t actually his voice in the video. Although the 36-year-old Argentine voice actor had worked for the channel over the past year, he learned in January 2023 that a voice generated by new artificial intelligence-powered software had replaced him. A technician at the YouTube channel laid out the stark truth for him: “It’s cheaper, and they don’t have to pay your rates,” Graue recalled to Rest of World.

On the day he found out he’d been replaced, Graue did what many people do — he sent out an angry tweet. Except his ire wasn’t directed at the companies building AI-powered voice-overs. Instead, he tweeted: “Thanks to all the actors and actresses who are lending their voices to create this shit that will eventually render all of us obsolete.”

Many workers in several industries have worried about being replaced by software, but in Latin America, it has actually started to happen. Voice actors have recently been reporting that they are losing out on jobs because they’ve been replaced by AI. Across Latin America, workers in the dubbing sector fear the disappearance of not just an entire industry, but an art form that has many fans. An increasingly popular option for voice actors is to take up poorly paid recording gigs at AI voice-over companies, training the very technology that aims to supplant them.

Many companies offer AI dubbing in Spanish, though few of those making inroads in Latin America are originally from Spanish-speaking countries. Each offers a slightly different twist on the art of dubbing: Tel Aviv-based Deepdub promises synthetic voices that replicate the whole range of human emotions, suitable for TV series and movies, while London-based Papercup targets nonfiction content for the likes of the BBC, Sky News, and the Discovery Channel. Seoul-based Klling, for its part, told Rest of World it combines AI dubbing with deepfake technology, where actors lip-sync to computer-generated voices. Despite their differences, all of these companies publicly defend their product as a cheaper and faster alternative to human voice actors. 

Oz Krakowski, chief revenue officer at Deepdub, told Rest of World that when it comes to dubbing Hollywood hits into other languages, AI voices will have an edge over human actors. He said that AI could eventually allow for a character voiced by a celebrity — say, Morgan Freeman — to retain their original voice while speaking flawless Spanish in any local accent and dialect.

Dubbing actors and their fans in Latin America are unimpressed by this claim. Gabriela Guzmán, a Mexican voice actor, rebutted it, telling Rest of World that “maybe [AI dubbing] can match the voice, but there is no way they can perform like the original actor. There is no soul, period.” Beyond the abstractions of soul, local dubbing actors serve a film far beyond their onscreen performance. Referring to movie marketing campaigns, Guzmán asked, “How do you take an AI voice to a Comic-Con to speak with its audience? There is no way … Voice actors have fans.”

Despite the faceless nature of AI-powered voice-over software, it’s ultimately all reliant on the work of humans because the software is trained using real performances. That can be done by hiring people to train the AI models on replicating a certain accent or voice. Alongside a technical team, the actor coaches the AI on the skills required to build an artificial voice from thousands of often random and meaningless recorded fragments. “For hours, they make us record a load of words and loose letters in different styles and tones,” a voice actor, who asked to remain anonymous for fear of being blacklisted from AI recordings, told Rest of World, saying the coaching involved “marathon sessions.”

Graue said that casting calls to record for AI companies started to appear in his hometown of Buenos Aires a few months ago. “When they offered me the gig, they told me that I had to record 10,000 words and that they would pay me 10,000 pesos [around $52 at the official exchange rate],” he recalled. Two sources in the industry, who requested anonymity for fear of losing work, confirmed to Rest of World that Buenos Aires is one of the places where European and Asian AI companies have been extracting voice data. They suspect the favorable exchange rate, which makes hiring people to train AI relatively cheaper in the country, may be related to the boom.

AI dubbing companies argue that the long-term savings they offer are even more substantial. Not only is recording in Argentina cheaper, but these human recordings are short-term expenditures. After recordings have been used by AI dubbing companies to create models in a proprietary “voice bank” or to train computer-generated voices that mimic the original ones, the AI firms claim that film and TV producers will need to pay them only for their copyrighted AI voices, instead of paying for translators, actors, sound mixing, and a recording studio.

An anonymous source also told Rest of World they’d been made to sign non-disclosure agreements by dubbing companies, even if they had been recording sounds simply for datasets, not meant for any specific content. According to this source, this is so that actors can’t publicly claim any part of a company’s bank of copyrighted artificial voices as their own.

Other methods of data collection are available to AI dubbing companies to complement recordings with actors. The “voice matching” method trains an AI on a film’s original voices. This is how AI software could potentially learn the vocal and tonal attributes of A-list Hollywood actors in order to translate and adapt those into a different language. Deepdub’s Krakowski told Rest of World that for this method, they only need two or three minutes of the original actors’ voice data.

Deepdub was the first company to dub an entire feature film into Spanish using AI: Every Time I Die, a thriller from the U.S., released in 2019. It is available in AI-generated español neutro — a spoken dialect that exists only in dubbed TV and film, meant to represent a version of “Neutral Spanish” to cater to a culturally diverse market of around 550 million, mainly concentrated in Mexico, Colombia, Argentina, the United States, and Spain (though European Spanish has its own “neutral” variant).

AI dubbing could eventually extinguish Neutral Spanish. Krakowski said “in the future, we will be able to diversify with more refined dialects and accents, which will allow us to create even more geographically specific content.”

But several Latin American voice actors are celebrities in their own right, crucial in pulling fans to conventions and driving sales at the box office. The dubbing industry connects local fans with global TV series and movies. They have also famously inserted culturally specific ad-libbed quips that go beyond the original scripted content. The community of voice actors plays a vital role in Latin American culture.

“We’ve listened to Latino Spanish [a variant of Neutral Spanish] since we were children,” Gonzalo Mori Molina, a Peruvian graphic designer, told Rest of World. “It created a special connection.” Mori Molina built a Latino Spanish dubbing wiki — an exhaustive database of voice actors and dubbing studios in Latin America. He particularly likes the fact that this artificial form of Spanish acts as a lingua franca across the region, allowing for a community to grow beyond national borders.

Currently, audiences also have issues with AI dubbing in less celebrity-driven content. The YouTube videos in which Graue, the Argentine voice actor, was replaced with an AI were received harshly by viewers. “Don’t ruin this with such horrible dubbing!” said one commenter. Another asked for Graue’s voice to be “reinstated.” Some threatened to stop watching the channel’s content altogether.

But the companies driving the AI-dubbing industry are confident that audiences will eventually come around. Moreover, Phil Newsom, head of production at MiLa Media, the studio behind Every Time I Die, told Rest of World that those in charge of getting feature films to viewers — the film distributors — don’t care who does the dubbing, whether it be a human or a machine.

Cynthia Alfonzo, a voice actor and a representative of Mexico’s National Association of Actors (ANDA), which includes all actors’ unions in Latin America, thinks there’s not much to be done if streaming and other content platforms adopt AI dubbing. “We might eventually release a statement underscoring the danger this [technology] poses and to inform about the harm it could do to actors,” Alfonzo told Rest of World.

It is still early days, though, and human actors and AI dubbing are still interacting in strange and unexpected ways. For instance, last year, the series Obi-Wan Kenobi premiered on Disney+. The English version used an AI-generated voice, developed by Ukrainian company Respeecher, for the iconic Darth Vader, recreating James Earl Jones’ signature gravelly vocal performance. However, the Latin American Spanish version was voiced by Argentine-Mexican actor Sebastián Llapur, who has played the role since 2015 — a human dubbing an AI performance.

Llapur told Rest of World that “nostalgia” was what drove the producers to clone Jones’ voice. “It’s hard to recast well-established characters.” However, he said that while AI technology was sometimes being implemented for artistic reasons in the U.S., it was clear to him that in Latin America, “[companies] are just trying to make something that’s already very cheap, even cheaper.”