Synthesia announces platform update with interactive AI videos, full-body avatars

Today, London-based Synthesia, a startup that enables enterprises to create professional-grade AI videos, announced a major update for its platform, aimed at providing a well-rounded suite for accelerating their video-first communication initiatives. 

Officially dubbed Synthesia 2.0, the update introduces several key capabilities, including full-body avatars capable of making a range of motions and an interactive video experience that will allow users to create AI videos with elements users can engage with, like a calendar or form. It also announced a new AI screen recorder that can simplify how companies create how-to videos for their workforce, among other content. 

The development follows the announcement of expressive avatars from Synthesia. However, it is important to note that not all features will debut right away. Some capabilities will launch next month, while others will roll out over the coming months. 

Next step in enhancing enterprise communications

Back in 2017, a team of AI researchers and entrepreneurs from Stanford, Technical University of Munich and Cambridge came together to start Synthesia. The goal was simple: give businesses a fast-tracked way to move away from monotonous text-based content to more engaging and captivating video content. Over the years, they developed an end-to-end platform where enterprises can create custom AI voices and avatars (they can even choose from existing ones) and combine them with pre-written or AI-produced scripts to generate AI videos.

Fast forward to today: Synthesia has been adopted by more than 55,000 businesses, including Zoom, Dupont, Heineken and Electrolux. The company has also extensively enhanced its AI avatars, making them more realistic and emotive. Just a few weeks ago, it debuted a new Express-1 model that enables the avatars to understand the context and sentiment in a piece of text and change their tone and facial expressions to deliver the speech.

With the latest update, the company is continuing the work on its avatars. Essentially, to enhance the storytelling aspect of the digital characters, the company is expanding their range of motion. This will enhance the personalities of the avatars, enabling them to tell captivating stories by using the full range of body language available to humans, including their hands.

According to Dan-Vlad Cobasneanu, Synthesia’s head of product marketing, the improved avatars are the outcome of capturing data from thousands of people worldwide to train several large video and audio foundation models. He added these avatars will also be fully controllable: users will be able to specify avatar appearance with images and videos and create animations with skeleton sequences.

But that’s just one part of the avatar upgrade.

Synthesia is also enhancing how users create their personal AI avatars by allowing them to use their webcams or mobile cameras with natural backgrounds. This, CEO Victor Riparbelli says, will be particularly useful in cases when the user wants to look more authentic, like when delivering a tutorial. The personal avatars recorded will also have better lip synchronization and a more natural voice, with the ability to translate voice into over 30 languages.

Interactive AI videos

While the improved avatars will enhance how the content is delivered, the new interactive video player built by Synthesia will change how it is consumed. Users will be able to integrate various clickable hotspots into their content, allowing end viewers to click and take action. For instance, they could click on an element to fill out a form, open a calendar/quiz, or navigate to only that part of the video they want to see. 

The feature still appears a few months away but the demo video did show that the user could enable these clickable experiences by simply enabling interactibility and defining a flow of where the hotspots would direct to. The first feature to debut in the suite of interactive experiences would be the ability to change the language and the displayed content of the video into the desired language, the company noted.

Notably, Synthesia is also adding an AI screen recorder. At first, the feature will work like a regular screen recorder, capturing everything happening on the screen. Once the recording is stopped, the underlying models of the company will generate a professional-grade AI video from it, complete with the audio of the speaker and the transcription of the audio. This can then be edited by the user to add their avatar and automatic zoom effects to emphasize key actions. They can even edit the script to update the content if needed.

What else comes with Synthesia 2.0?

Among other things, Synthesia 2.0 is getting some incremental improvements, including the ability to add brand kits (to incorporate their brand language and identity in the videos) and generate content in bulk via the company’s AI-powered video assistant. 

There will also be new collaboration capabilities allowing multiple users to work on video projects at the same time and an improved one-click translation experience, where users would have to create and maintain just one version of a video. The translations will be done and updated automatically. 

It will be interesting to see how these new capabilities boost the adoption of Synthesia, which has heavily focused on enterprise applications with a consent, moderation and collaboration-driven approach. Other players competing with the company in this space are Deepbrain AI, Rephrase and HeyGen.