Osegi
AI·2 min read

Sber Open-Sources Kandinsky Avatar 1.0 Talking Avatar Model

Sber has released the code and weights of Kandinsky Avatar 1.0 under the MIT license. The model turns an image and a speech recording into an animated talking character.

Animated talking avatar generated from a single portrait image and an audio recording by an AI model.
Image: Хабр
SUMODialoediviewergiftdetect

Sber's developers have released Kandinsky Avatar 1.0, a model that takes an image and a speech recording and produces an animated talking character. The code and weights are open and available under the MIT license.

The model syncs lip movements with speech and adds facial expressions, gestures and head turns. Speaking style and desired emotions can be set through a text prompt. The avatar can be a person, a cartoon character or an animal. It was trained on top of Kandinsky 5.0 Video using 1.17 million clips with a single speaker in frame, and the dataset covers 95 languages.

Two sizes and hardware needs

There are two versions: Lite, with 2 billion parameters, generates SD video, while Pro, with 21 billion parameters, generates HD video. Sber recommends Nvidia GPUs with 48 GB of memory for SD and 80 GB for HD. Decoding frames in parts cuts peak memory use to 28 GB, but visible seams between segments may then appear in the video.

Benchmark results

Kandinsky Avatar 1.0 Pro was compared with leading open and proprietary models in a blind test. Participants watched pairs of clips, rated them on seven criteria and picked the better one without knowing which model made it. Pro's videos were chosen more often than each rival's: in 63% of cases against Kling Avatar 2, 60% against LongCat-Avatar-1.5 and 54% against HeyGen IV.

Why it matters

Video avatars can be used for training and advertising clips, video instructions and personalized messages to customers. Denis Dimitrov, managing director for data research at Sberbank, said the model is being opened so Russian developers have their own technological foundation for such solutions, one they can embed in products and fine-tune without building the technology from scratch. Community feedback will inform future Kandinsky versions and the rollout of video avatars in Sber's services.

Code and weights are free on GitHub, GitVerse and Hugging Face. The model is also expected to be usable in ComfyUI and Diffusers.

Apps worth a look

SUMO: the number puzzle

Tap the right tiles, add them up, hit the target. Easy to learn, hard to master. Free, for all ages.

Tags:sberkandinskyavataropen sourcevideo generationmit license

More from this day

See all →

More in: AI

See all →