Video-Motion-Customization
Gen-L-Video
Video-Motion-Customization | Gen-L-Video | |
---|---|---|
3 | 1 | |
119 | 260 | |
- | - | |
7.2 | 7.7 | |
about 1 month ago | 4 months ago | |
Python | Jupyter Notebook | |
Apache License 2.0 | Apache License 2.0 |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
Video-Motion-Customization
- Code for video motion customization has been released!
-
VMC: Video Motion Customization
Text-to-video diffusion models have advanced video generation significantly. However, customizing these models to generate videos with tailored motions presents a substantial challenge. In specific, they encounter hurdles in (a) accurately reproducing motion from a target video, and (b) creating diverse visual variations. For example, straightforward extensions of static image customization methods to video often lead to intricate entanglements of appearance and motion data. To tackle this, here we present the Video Motion Customization (VMC) framework, a novel one-shot tuning approach crafted to adapt temporal attention layers within video diffusion models. Our approach introduces a novel motion distillation objective using residual vectors between consecutive frames as a motion reference. The diffusion process then preserves low-frequency motion trajectories while mitigating high-frequency motion-unrelated noise in image space. We validate our method against state-of-the-art video generative models across diverse real-world motions and contexts. Our codes, data and the project demo can be found at https://video-motion-customization.github.io/
Code: https://github.com/HyeonHo99/Video-Motion-Customization
Gen-L-Video
What are some alternatives?
MotionDirector - MotionDirector: Motion Customization of Text-to-Video Diffusion Models.
LAMP - Official implement code of LAMP: Learn a Motion Pattern by Few-Shot Tuning a Text-to-Image Diffusion Model (Few-shot-based text-to-video diffusion)
TokenFlow - Official Pytorch Implementation for "TokenFlow: Consistent Diffusion Features for Consistent Video Editing" presenting "TokenFlow" (ICLR 2024)
storyteller - Multimodal AI Story Teller, built with Stable Diffusion, GPT, and neural text-to-speech
Awesome-Video-Diffusion - A curated list of recent diffusion models for video generation, editing, restoration, understanding, etc.
Wuerstchen - Official implementation of Würstchen: Efficient Pretraining of Text-to-Image Models
anima - Turn text into video using Stable Diffusion and Google FILM
MultiDiffusion - Official Pytorch Implementation for "MultiDiffusion: Fusing Diffusion Paths for Controlled Image Generation" presenting "MultiDiffusion" (ICML 2023)