Umang Agarwal is a Hrithik Roshan superfan. For the past five years, he’s run a YouTube channel dedicated to the superstar, who’s popularly dubbed “the Greek god of Bollywood.” Aggarwal typically reposts videos of Roshan’s dance performances, personal messages to politicians and actors, and behind-the-scenes shoots at exotic locations. During the Hindu festival of Rakhi this August, Agarwal shared a remarkable video.
“Hello Umang,” Roshan said in the 16-second clip. “Happy Rakhi to you.” The video was viewed thousands of times. Followers of the channel couldn’t believe Aggarwal had solicited a greeting from their favorite actor: “Brother, you are very lucky,” one person said. Another commented that his years of “hard work” running a YouTube fan channel for the superstar “paid off.”
Aggarwal wasn’t the only person to get a personal message from Roshan. Across social media, fans were getting the same, addressed by name. They weren’t real, not exactly. They were deepfakes, created for the confectioner Cadbury by Rephrase.ai, a Bengaluru-based startup that is pioneering the commercial use of artificial intelligence–generated avatars based on real people. Roshan had licensed the rights to his image to Cadbury, allowing the company, with Rephrase.ai’s help, to make him say whatever they wanted. Users simply had to buy a limited edition chocolate box, scan a QR code, and enter a name for the actor to say.
“I couldn’t believe it was true,” Aggarwal said. “I sent it to everybody [in my friends and family circle]. They were all very happy.”
Deepfake tech, which allows a user to create highly realistic simulations of a real person, has been widely associated with pornography and fears of political misinformation. (Rephrase.ai distances itself from the term “deepfake,” and calls its technology “facial reenactment”). But it’s steadily gaining a toehold in the corporate world. Rather than going through the arduous process of finding a film crew, booking an actor, hiring expensive equipment, and spending time on postproduction, brands are trying out platforms where they can simply type in the script to create realistic AI-generated videos. They can use the no-code platform to personalize sales processes, make corporate training videos, and leave audiences with a lingering emotional connection through personalized celebrity ads. Rephrase.ai counts among its clients Amazon-owned retailer Zappos, Indian financial services giant Bajaj Finserv, and the American home improvement retailer Lowe’s.
Deepfakes can be seductive for marketers: they’re cheap and easy to make, especially as Covid-19 limits in-person meetings and makes traditional shoots risky to execute. They’re getting ever more convincing and ever more affordable. The technology could usher in a new era of advertising, but to get there, brands, suppliers like Rephrase.ai, consumers, and regulators will have to navigate a tricky ethical and legal landscape.
Ashray Malhotra, Rephrase.ai’s co-founder, smiles often and speaks fast, switching occasionally to Hindi while transitioning between sentences. While studying at the Indian Institute of Technology in Mumbai he convened the school’s film club and made his own short films.
“If you’ve made professional-quality films, you realize how painful that process is,” he told Rest of World. “It doesn’t need to be that way.”
In 2015, while still at college, Malhotra and Nisheeth Lahoti, his co-founder at Rephrase.ai, made a piece of software that corrected actors’ lip movements in dubbed movies. But the market was limited to a small number of dubbing studios and production houses, so they decided to branch out further. The pair, along with college friend and former Facebook engineer Shivam Mangla, founded Rephrase.ai in 2018. They reworked the core video engine from their dubbing product so that it could create audio-visual content from scratch.
In 2019, they were accepted into a Techstars accelerator program. On the demo day, they showed off a video of President Barack Obama speaking in the voice of Amazon’s Alexa.
Their pitch is that they can create professional-quality videos using just text. They have around 25 off-the-shelf A.I. models of different ethnicities that companies can choose. These are based on “deepfake actors” — the terminology is still to be settled — who are like stock photo models, real people who have licensed their likenesses. They’ve been assigned pseudonyms — Jane, Sanjana, Anya — to protect their identities. But they also offer clients a bespoke service: custom models of celebrities, executives, or influencers, who can be re-created in AI for use in advertising, sales, or training.
“If you’re a CXO at a Fortune 500 company [with a] 10,000-member sales team, of which 7,000 of them hit their sales quotas, this now gives you a chance to do a one-on-one interaction with every single person who hit their quotas and congratulate them,” said Malhotra.
“This is going to become mainstream in the next six months.”
Rephrase.ai has created 45-minute-long lectures featuring custom faces for the accounting firm PwC. The company is also working with an Indian beauty brand to let it run multiple, customized versions of the same ad. “Something that was impossible before because there’s no way you can get time from your brand ambassador to shoot 100 different videos,” Malhotra said.
He has recently signed a deal with Collective Artist Network, formerly called KWAN, India’s largest talent management company, to broaden the use of the technology. At the moment, celebrities working for brands tend to do one-off shoots for adverts. In the future, deepfakes will let them sign up to be part of bigger, more intricate customer acquisition and retention campaigns.
“This is going to become mainstream in the next six months,” Malhotra said. They’ve already begun working on a few celebrity-led personalized ads for the Hindu festival of Diwali.
One early adopter of the product is entrepreneur and online influencer Ankur Warikoo, who has built a reputation as a productivity guru. He shares pithy, piece-to-camera-style videos, usually 20 minutes or less long, with his 3 million followers on social media, offering career and personal finance advice. He makes most of his revenue through online courses — he’s sold 32,000 copies of his time-management course — and uses social media as a way to build trust with his audience.
In August, Warikoo signed up to use Rephrase.ai to create an AI double. He recorded a master video from his home office, in front of a framed poster reading “DO EPIC SHIT.” Rephrase’s algorithm needs a 20-minute input video, shot in high definition, allowing the system to capture facial movements, track pronunciation, and break down words into the phonemes it needs to be able to create speech from scratch. There’s a one-time fixed cost to creating the digital avatar, afterward, there’s a fee per video.
Warikoo said the process wasn’t vastly different from his usual videos. “The big difference was, I’m usually very animated with my hands, but I couldn’t use them anymore,” he said. Rephrase.ai informed him that any bodily movement in the input video could break the algorithm. He recorded a video with his head held straight and uploaded the footage shot in 4K onto a Dropbox folder and, a week later, received a notion document containing his digital avatar.
AI Warikoo read out scripts, both old and new, pronouncing tricky names that he had never attempted to pronounce. The clip, he said, “literally blew my mind.” He’s already begun using the avatar to offer personalized messages for anyone who buys his course. When users pay through his landing page, the system fetches the buyer’s first name and Rephrase.ai renders a video of Warikoo welcoming them by name, which is sent over his WhatsApp Business account.
Warikoo said he hopes that the personalized videos “will create this wow factor for something which is largely considered to be a transactional exchange.” The prerecorded courses have no live interactions, and a personalized welcome might “elevate the mental view of what this course is. Now, will that result in more sales? More word of mouth? More people sharing it? I don’t know.”
So-called “synthetic media” companies, which produce AI-generated content, have attracted over $1.5 billion in investments since 2016, according to market intelligence company CB Insights. Competition in the space is growing. London-based Synthesia recently created a personalized ad campaign featuring football legend Lionel Messi.
CB Insights’ AI for digital content report said that 2021 is set to be a record year for investments in synthetic media, although it noted that despite the rush of interest in startups, Big Tech continues to dominate synthetic media research with Facebook, IBM, Snapchat, and Apple having each filed more than 15 avatar-related patents since 2016. Rephrase.ai is backed by Silicon Valley investors Lightspeed Venture Partners and AV8 Ventures.
With more money flowing into the sector, the quality of the deepfake productions has improved. When Abhay Tandon, the head of Lowe’s Innovation Labs in India, the research arm of the American retailer, tried out the tech in September 2020, the algorithm couldn’t deal with his beard, spectacles, and head movements. He had to shave before he shot the master video — “the cost of experimenting,” Tandon said. Less than a year later, when he recorded a new video, the algorithm was able to accommodate his facial hair and even some hand gestures. This time around, “there were no outright issues at all. It was completely seamless,” Tandon said.
However, Tandon said he felt that despite the improvements in the way it speaks and expresses emotions, his AI avatar still has some way to go “to look completely human-like.”
Lowe’s is trying to figure out a use case for the technology and, importantly, whether potential customers will be able to overcome the “uncanny valley” and trust AI-generated avatars. As corporate deepfakes become more realistic and accessible, their use cases will increase, but so will the ethical challenges they present.
The rapid development of the technology means that it’s running ahead of regulations and laws, according to Henry Ajder, a consultant who advises organizations on deepfakes and other developments in artificial intelligence. “Synthetic media is disruptive — in a way that the law is just not quite currently equipped to deal with the kind of gray cases and edge cases that it raises,” he said.
With no established standards around how deepfakes should be sold and distributed, companies are having to self-regulate. Rephrase.ai’s ethics policy states: “We will not offer our software for widespread consumer use.” It also states that its technology will be provided to only “clients who align” with its “ethical code of conduct.” Synthetic videos created by Rephrase.ai come with an “assured by Rephrase” watermark, signifying that they’re manipulated, but not explicitly stating it.
“Earlier, you only had your pictures; limited stuff can be done with it. Now you have your video out there, which can be distorted.”
At the moment, image licensing in commercial deepfake projects exists in a gray area. For instance, Lowe’s paid for the creation of Tandon’s AI avatar. Tandon is an employee, who volunteered for the project. “The current model — as long as I choose to give consent to Lowe’s in utilizing it — is Lowe’s property,” Tandon said. “If I choose to move out of Lowe’s, they cannot create any more videos without my consent.” Tandon said that he is allowed to remove videos and to revoke consent for the creation of future videos.
These gray areas are more concerning for women, who are far more likely to be the victims of malicious use of synthetic media. LinkedIn influencer and tech entrepreneur Preksha Kaparwan said that her photos have previously been used to create fake accounts on the dating platform Tinder, which made her hesitate before cloning herself on Rephrase.ai’s system.
“Earlier, you only had your pictures; limited stuff can be done with it. Now you have your video out there, which can be distorted,” she told Rest of World. In the end, she signed up anyway, sharing a video to her 200,000-strong LinkedIn community, receiving an encouraging response. She plans to use the avatar for walkthrough videos for her own software product, Alphaa AI.
Solving these issues of ownership, and addressing the potential for abuse, could be a major challenge for the companies trying to usher synthetic media into the mainstream. But it could also be an opportunity for them to impose standards and to rehabilitate a technology that has suffered from association with unethical use.
“I think we’re at this inflection point,” said Sam Gregory, program director at human rights organization Witness, which studies the use of deepfakes. Companies need to create policies covering informed consent for those giving over their image rights, proper labeling of synthetic media, and ways to make sure that synthetic videos are forensically detectable, he said. “There’s actually a really critical role [for] corporate deepfake makers … to demonstrate how they’re adhering to these principles and to actually start doing it in an industry context.”
