Video-Motion-Customization
Awesome-Video-Diffusion
Video-Motion-Customization | Awesome-Video-Diffusion | |
---|---|---|
3 | 1 | |
117 | 2,478 | |
- | 9.3% | |
7.2 | 8.9 | |
about 1 month ago | 9 days ago | |
Python | ||
Apache License 2.0 | - |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
Video-Motion-Customization
- Code for video motion customization has been released!
-
VMC: Video Motion Customization
Text-to-video diffusion models have advanced video generation significantly. However, customizing these models to generate videos with tailored motions presents a substantial challenge. In specific, they encounter hurdles in (a) accurately reproducing motion from a target video, and (b) creating diverse visual variations. For example, straightforward extensions of static image customization methods to video often lead to intricate entanglements of appearance and motion data. To tackle this, here we present the Video Motion Customization (VMC) framework, a novel one-shot tuning approach crafted to adapt temporal attention layers within video diffusion models. Our approach introduces a novel motion distillation objective using residual vectors between consecutive frames as a motion reference. The diffusion process then preserves low-frequency motion trajectories while mitigating high-frequency motion-unrelated noise in image space. We validate our method against state-of-the-art video generative models across diverse real-world motions and contexts. Our codes, data and the project demo can be found at https://video-motion-customization.github.io/
Code: https://github.com/HyeonHo99/Video-Motion-Customization
Awesome-Video-Diffusion
What are some alternatives?
MotionDirector - MotionDirector: Motion Customization of Text-to-Video Diffusion Models.
awesome-speech-recognition-speech-synthesis-papers - Automatic Speech Recognition (ASR), Speaker Verification, Speech Synthesis, Text-to-Speech (TTS), Language Modelling, Singing Voice Synthesis (SVS), Voice Conversion (VC)
TokenFlow - Official Pytorch Implementation for "TokenFlow: Consistent Diffusion Features for Consistent Video Editing" presenting "TokenFlow" (ICLR 2024)
video-diffusion-pytorch - Implementation of Video Diffusion Models, Jonathan Ho's new paper extending DDPMs to Video Generation - in Pytorch
SpeechT5 - Unified-Modal Speech-Text Pre-Training for Spoken Language Processing
storyteller - Multimodal AI Story Teller, built with Stable Diffusion, GPT, and neural text-to-speech
ReuseAndDiffuse - Reuse and Diffuse: Iterative Denoising for Text-to-Video Generation
Gen-L-Video - The official implementation for "Gen-L-Video: Multi-Text to Long Video Generation via Temporal Co-Denoising".