CVPR2022-DaGAN
wunjo.wladradchenko.ru
Our great sponsors
CVPR2022-DaGAN | wunjo.wladradchenko.ru | |
---|---|---|
5 | 6 | |
936 | 694 | |
- | - | |
5.8 | 9.5 | |
5 months ago | 3 days ago | |
Python | Python | |
GNU General Public License v3.0 or later | MIT License |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
CVPR2022-DaGAN
-
DaGAN++: Depth-Aware Generative Adversarial Network for Talking Head Video Generation
Predominant techniques on talking head generation largely depend on 2D information, including facial appearances and motions from input face images. Nevertheless, dense 3D facial geometry, such as pixel-wise depth, plays a critical role in constructing accurate 3D facial structures and suppressing complex background noises for generation. However, dense 3D annotations for facial videos is prohibitively costly to obtain. In this work, firstly, we present a novel self-supervised method for learning dense 3D facial geometry (ie, depth) from face videos, without requiring camera parameters and 3D geometry annotations in training. We further propose a strategy to learn pixel-level uncertainties to perceive more reliable rigid-motion pixels for geometry learning. Secondly, we design an effective geometry-guided facial keypoint estimation module, providing accurate keypoints for generating motion fields. Lastly, we develop a 3D-aware cross-modal (ie, appearance and depth) attention mechanism, which can be applied to each generation layer, to capture facial geometries in a coarse-to-fine manner. Extensive experiments are conducted on three challenging benchmarks (ie, VoxCeleb1, VoxCeleb2, and HDTF). The results demonstrate that our proposed framework can generate highly realistic-looking reenacted talking videos, with new state-of-the-art performances established on these benchmarks. The codes and trained models are publicly available on the GitHub project page at https://github.com/harlanhong/CVPR2022-DaGAN
-
Animating generated face test
I use https://github.com/harlanhong/CVPR2022-DaGAN it's supposedly faster than TPSMM.
-
Using SD to make 'deepfakes' demo
Picture to Animation : Depth-Aware Generative Adversarial Network for Talking Head Video Generation (CVPR 2022) https://github.com/harlanhong/CVPR2022-DaGAN This gave me Picture to Animation.
-
Waifu diffusion - reanimation with DaGAN
Thanks, I will take a look. Do you have any more info on the image segmentation part, was looking through the github and could not find anything, only on face alignment: https://github.com/harlanhong/CVPR2022-DaGAN/tree/master/face-alignment
wunjo.wladradchenko.ru
-
Check out Wunjo AI – open-source AI Toolkit
AI Retouch Tool & Segmentation Mask
GitHub Stars Needed!
We're at 499 stars on GitHub, just 13 away from a cool milestone! If you like what you see, I'd appreciate your support. Check it out and drop a star if you find it interesting.
GitHub Repository: https://github.com/wladradchenko/wunjo.wladradchenko.ru
Thanks a bunch for your time and support!
-
Update Wunjo AI:Voice cloning from music, remove text from video, more deepfakes
An update has been released for Wunjo AI 1.6.1, featuring voice cloning, deepfake creation, and video-to-video transformation by prompt. Now, you can clone a voice from a song and remove lyrics from a video. Open-source code available at https://github.com/wladradchenko/wunjo.wladradchenko.ru and article about the update on https://dev.to/wladradchenko.
- Voice Cloning, Face Swap, Video Object Removal: Open-Source Wunjo AI on Python
-
Unleash Creativity with Wunjo AI: Synthesize Speech and Create Deepfake Videos on free open-source project
Ready to dive in? Visit our GitHub repository at GitHub to explore the code, documentation, and resources. Whether you're a seasoned developer or simply curious about AI, you'll find a welcoming environment to experiment and contribute. Also you can install on Linux, MacOS or Windows.
What are some alternatives?
Thin-Plate-Spline-Motion-Model - [CVPR 2022] Thin-Plate Spline Motion Model for Image Animation.
GeneFace - GeneFace: Generalized and High-Fidelity 3D Talking Face Synthesis; ICLR 2023; Official code
sd-wav2lip-uhq - Wav2Lip UHQ extension for Automatic1111
Voice-Cloning-App - A Python/Pytorch app for easily synthesising human voices
Face-Depth-Network - The component of DaGAN (CVPR 2022)
chatgpt-voice-assistant - A voice assistant powered by OpenAI's ChatGPT language model, currently available in six languages.
PaddleGAN - PaddlePaddle GAN library, including lots of interesting applications like First-Order motion transfer, Wav2Lip, picture repair, image editing, photo2cartoon, image style transfer, GPEN, and so on.
awesome-talking-head-generation
ControllableTalkNet - This is a modified version of NVIDIA's TalkNet. It is a controllable network that can be used for both CPU and GPU inference.