Osegi
AI·2 min read

Sber Releases Kandinsky Avatar 1.0 as Open-Source Talking Avatar AI

Sber has opened access to Kandinsky Avatar 1.0, a model family that turns a photo and a voice recording into talking video avatars up to one minute long.

A realistic digital avatar of a speaking person generated from a photo and voice recording by a neural network.
Image: rb.ru
SUMODialoediviewer

Sber has opened access to Kandinsky Avatar 1.0, a family of neural networks that generates talking video avatars from a single photo and an audio recording. The models produce clips of up to one minute at 24 frames per second, the company said.

The system syncs lip movement with speech and adds facial expressions, head turns and gestures on its own. To make a video, a user supplies a photo, an audio track and a short text description of the scene. Emotions such as joy, sadness or anger can be set with a text prompt. Speech plays in the language of the source recording. Kandinsky Avatar does not generate or clone voices, so the audio must be provided separately.

Open code under the MIT license

Code and weights are published under the MIT license on GitHub, GitVerse and Hugging Face. Developers can download them for free and use them in their own projects, including commercial ones. Sber also plans support in popular tools for working with third-party neural networks, and intends to bring video avatars to its own services.

The company lists several uses: tools for bloggers with virtual presenters, educational platforms with digital teachers, virtual customer consultants, video localization with synchronized lip movement, game and animation characters, and personalized video greetings.

Pro and Lite versions

Kandinsky Avatar is built on the Kandinsky 5.0 Video generation model and comes in two versions. Pro outputs HD video. In Sber's internal comparisons, it matches the quality of the commercial service HeyGen and beats the open alternatives tested. Lite targets weaker hardware and can run on a home graphics card, but produces SD-resolution video.

The models were trained on more than a million video clips, including interviews, talks and podcasts in 95 languages such as Russian, English, Kazakh, Spanish and French.

"We are opening the model so that Russian developers have their own technological foundation for such solutions," said Denis Dimitrov, managing director for data research at Sberbank. He said developers can embed it in products and fine-tune it for specific tasks without building the technology from scratch. Sber expects developer feedback and plans to use it to improve future versions of Kandinsky.

Apps worth a look

SUMO: the number puzzle

Tap the right tiles, add them up, hit the target. Easy to learn, hard to master. Free, for all ages.

Tags:sberkandinsky avataropen sourcevideo generationhugging faceavatars

More from this day

See all →

More in: AI

See all →