With global content consumption rising and demand for non-English content surpassing that for English movies and shows, IMAX is leveraging AI to scale localization on its original content.
The entertainment and media industry grew 5% to $2.8 trillion in 2023, according to a report by PwC. The industry is expected to continue its expansion, though at a modest compound annual growth rate of nearly 4% to $3.4 trillion over the next five years. Non-English language content is growing rapidly even in English markets, including the U.S., U.K., Australia, and Canada. Last year, Netflix reported a 90% growth in its viewership of non-English content in the U.K. over the last three years.
IMAX has taken all this into account and is now exploring localization using AI to get more eyeballs.
On Monday, the Canadian production theater company, known for its massive theaters and immersive movie experiences, announced its partnership with Dubai-based startup Camb.ai to use its AI speech models to translate original content, including documentaries.
Camb.ai, which has already deployed its AI dubbing and speech translation for live sports events and leagues, including the Australian Open, Eurovision Sport, and Major League Soccer, offers its Boli model for speech-to-text translation and Mars for speech emulation. The models are available through the startup’s DubStudio platform, which supports 140 languages, including various low-resource languages that do not have significant data on the internet.
“Companies like OpenAI and Anthropic have a different vision of society,” said Akshat Prakash, co-founder and CTO of Camb.ai, in an exclusive interview. “They’re trying to build models that are very horizontal and can cover a large breadth of tasks. … We don’t have to do that at all. Some of our models are less than 100 million parameters and are super specialized.”
Prakash, a former Apple engineer who worked with the team developing AI and ML models for Siri, co-founded Camb.ai with his father Avneesh Prakash last year.
“We are a generation apart, and growing up in India. Thirty years apart, we encountered the same language challenges,” the CTO told TechCrunch.
He said that Camb.ai pretrained 70% of its models using academic-licensed datasets, which are commercially usable. The remaining 30% involves fine-tuning data it gets through early partners who deploy its models for AI-based dubbing and translation.
“What we don’t do, and we’ve been very careful and completely avoiding that, is potentially scrapping the internet,” Prakash asserted. “Some companies feel that they can get away with it because they’re building a consumer-facing app or tool, and they believe it’s fine to scrape, like 10 petabytes of the internet.”
Camb.ai uses a “three-layer” approach to provide AI-based translation, comprising the foundation layer of its Boli and Mars models, the infrastructure layer that hosts these AI models, and then the DubStudio platform for the front end.
Unlike other AI-based models, Camb.ai’s Boli takes input speech tokens, produces output text tokens in the translated language, and retains nuances, Prakash claims. Once Boli generates the text, Mars translates the text into speech using the same audio input signal to capture the performance of the actual audio, including ambient sound, such as the background score of the audience cheering in the case of sports events, he said.
Prakash told TechCrunch that Camb.ai’s tech delivers speech translation in up to 10 languages simultaneously with 20 to 30 seconds of latency, which can be covered by the streaming and broadcasting delay of 30 to 40 seconds.
IMAX will roll out AI translations in stages, starting with high-resource languages. The deployment comes after internal testing of Camb.ai’s tech on its original content.
“While we are only in the beginning stages of the partnership, we will continue to work together to better explore its potential and how it can best move us forward,” said Mark Welton, president of IMAX Global.
Welton indicated that the AI deployment would help save on translation costs without disclosing specifics.
Camb.ai currently has a team of 50 people. In February, it raised $4 million in a seed round led by Courtside Ventures. Prakash told TechCrunch that the startup is closing a bigger, pre-Series A round to expand its reach and headcount.