Top 6 Python talking-head Projects

SadTalker

16 10,394 6.9 Python

[CVPR 2023] SadTalker：Learning Realistic 3D Motion Coefficients for Stylized Audio-Driven Single Image Talking Face Animation

Project mention: Can some expert analyze a github repo and tell us if it's really safe or not? | /r/cybersecurity | 2023-12-07

GeneFace

1 2,340 6.7 Python

GeneFace: Generalized and High-Fidelity 3D Talking Face Synthesis; ICLR 2023; Official code

Project mention: What is the best AI Lip Sync in 2023? | /r/singularity | 2023-05-05

There's GeneFace, but idk how good it is

WorkOS

workos.com sponsored

The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.
sd-wav2lip-uhq

1 1,035 7.8 Python

Wav2Lip UHQ extension for Automatic1111

Project mention: Time Travel Scene - Gen-2, ElevenLabs, Wav2Lip | /r/aivideo | 2023-08-10

Hi, Great job, If you need to remove the wav2lip blur, you can use : https://github.com/numz/sd-wav2lip-uhq

CVPR2022-DaGAN

5 936 5.8 Python

Official code for CVPR2022 paper: Depth-Aware Generative Adversarial Network for Talking Head Video Generation

Project mention: DaGAN++: Depth-Aware Generative Adversarial Network for Talking Head Video Generation | /r/BotNewsPreprints | 2023-05-11

Predominant techniques on talking head generation largely depend on 2D information, including facial appearances and motions from input face images. Nevertheless, dense 3D facial geometry, such as pixel-wise depth, plays a critical role in constructing accurate 3D facial structures and suppressing complex background noises for generation. However, dense 3D annotations for facial videos is prohibitively costly to obtain. In this work, firstly, we present a novel self-supervised method for learning dense 3D facial geometry (ie, depth) from face videos, without requiring camera parameters and 3D geometry annotations in training. We further propose a strategy to learn pixel-level uncertainties to perceive more reliable rigid-motion pixels for geometry learning. Secondly, we design an effective geometry-guided facial keypoint estimation module, providing accurate keypoints for generating motion fields. Lastly, we develop a 3D-aware cross-modal (ie, appearance and depth) attention mechanism, which can be applied to each generation layer, to capture facial geometries in a coarse-to-fine manner. Extensive experiments are conducted on three challenging benchmarks (ie, VoxCeleb1, VoxCeleb2, and HDTF). The results demonstrate that our proposed framework can generate highly realistic-looking reenacted talking videos, with new state-of-the-art performances established on these benchmarks. The codes and trained models are publicly available on the GitHub project page at https://github.com/harlanhong/CVPR2022-DaGAN

wunjo.wladradchenko.ru

6 694 9.5 Python

Wunjo AI: Synthesize & clone voices in English, Russian & Chinese, real-time speech recognition, deepfake face & lips animation, face swap with one photo, change video by text prompts, segmentation, and retouching. Open-source, local & free.

Project mention: Check out Wunjo AI – open-source AI Toolkit | news.ycombinator.com | 2024-01-16

AI Retouch Tool & Segmentation Mask
GitHub Stars Needed!
We're at 499 stars on GitHub, just 13 away from a cool milestone! If you like what you see, I'd appreciate your support. Check it out and drop a star if you find it interesting.
GitHub Repository: https://github.com/wladradchenko/wunjo.wladradchenko.ru
Thanks a bunch for your time and support!

Text2Video

1 408 3.8 Python

ICASSP 2022: "Text2Video: text-driven talking-head video synthesis with phonetic dictionary".

NOTE: The open source projects on this list are ordered by number of github stars. The number of mentions indicates repo mentiontions in the last 12 Months or since we started tracking (Dec 2020).