SD-CN-Animation
RAFT
| SD-CN-Animation | RAFT | |
|---|---|---|
| 20 | 3 | |
| 802 | 3,028 | |
| - | 3.3% | |
| 8.0 | 3.2 | |
| 6 months ago | 6 months ago | |
| Python | Python | |
| MIT License | BSD 3-clause "New" or "Revised" License |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
SD-CN-Animation
-
AnimDiff
1) generate video using https://github.com/camenduru/animatediff 2) upscale using SD-CN https://github.com/volotat/SD-CN-Animation 3) interpolate frames using https://github.com/google-research/frame-interpolation 4) add audio using https://huggingface.co/spaces/suno/bark
- Doctor Strange Multiverse Maestro (SD-CN)
-
Controlnet Error: "No ControlNetUnit detected in args".
I removed two extensions I wasn't using that occur before it in alphabetical order (for video... SD-CN-Animation and Infinite Zoom ). I think it may have been the latter two which were the problem.
-
I've been playing with AI transformations and made these gifs. If anyone has any interesting suggestions, I'll see if I can make some of them. Done with 'Craiyon' and 'Dream by WOMBO'
they are probably using something like this in background https://github.com/volotat/SD-CN-Animation
-
Turning a viral NPC girl into an actual NPC. Made with SD-CN-Animation. Instruction in the comments.
Tool used: https://github.com/volotat/SD-CN-Animation
- Weekly Megathread - 14 May 2023
-
Experimenting with SD-CN-Animation extension
Extension: SD-CN-Animation Model: Breaking Domain Realistic Seed: 435324545
- volotat/SD-CN-Animation: This script allows to automate video stylization task using StableDiffusion and ControlNet.
-
New loopback animation method using FloweR + TemporalNet + LUFS
FloweR https://github.com/volotat/SD-CN-Animation/ Estimates a flow map out of last 4 for image (caveman txt2video)
-
Optical flow installation for a Stable Diffusion extension
The command "conda install pytorch=1.6.0 torchvision=0.7.0 cudatoolkit=10.1 matplotlib tensorboard scipy opencv -c pytorch" is asking to install Google Authenticator in an environment, as part of a package install. Trying to install this https://github.com/princeton-vl/RAFT to run this https://github.com/volotat/SD-CN-Animation
RAFT
-
SD-CN-Animation updated to v0.9. Many of the major issues were fixed, better vid2vid quality, ControlNet in txt2vid mode and more! Here is an example of how txt2vid with CN looks like.
Learning something new. Recently learned to use temporal kit, i.e. merge multiple frames into the same image to achieve consistency. You are using " 'RAFT' optical flow estimation algorithm to create an occlusion mask that is used to generate the next frame". Curious, how is it used?
-
Optical flow installation for a Stable Diffusion extension
The command "conda install pytorch=1.6.0 torchvision=0.7.0 cudatoolkit=10.1 matplotlib tensorboard scipy opencv -c pytorch" is asking to install Google Authenticator in an environment, as part of a package install. Trying to install this https://github.com/princeton-vl/RAFT to run this https://github.com/volotat/SD-CN-Animation
-
Are the New M1 Macbooks Any Good for Deep Learning? Let’s Find Out
https://github.com/princeton-vl/RAFT#optional-efficent-imple...
It's "optional" in the sense that things still calculate correctly on CPU without it, but at a 1000x performance penalty. Or you could skip it if you had 64GB of GPU RAM, which you cannot buy (yet).
So if you actually want to work with this on GPUs that are commercially available, you need it.
What are some alternatives?
AnimateDiff - Official implementation of AnimateDiff.
tensorflow_macos - TensorFlow for macOS 11.0+ accelerated using Apple's ML Compute framework.
infinite-zoom-automatic1111-webui - infinite zoom effect extension for AUTOMATIC1111's webui - stable diffusion
Pointnet_Pointnet2_pytorch - PointNet and PointNet++ implemented by pytorch (pure python) and on ModelNet, ShapeNet and S3DIS.
AnimateDiff - Official implementation of AnimateDiff.
ROCm - AMD ROCm™ Software - GitHub Home [Moved to: https://github.com/ROCm/ROCm]
private-gpt - Interact with your documents using the power of GPT, 100% privately, no data leaks
pointnet2 - PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space
llama.cpp - LLM inference in C/C++
llm - An ecosystem of Rust libraries for working with large language models
frame-interpolation - FILM: Frame Interpolation for Large Motion, In ECCV 2022.
SecondShiftAugie - Second Shift Augie is a sassy and sarcastic AI assistant that helps answer questions and summarize YouTube videos. It has several different features, including text-to-speech functionality, and the ability to answer questions about its own code with the !selfreflect command.