dain-ncnn-vulkan VS stable-diffusion-videos

Lien vers les instructions [https://github.com/nihui/dain-ncnn-vulkan/blob/master/readme.md de Readme.md)

I also found this and this, which FlowFrames seems to be based on. These would probably help with the frame extrapolation and increasing, but won't help me with the upscaling.

not SD based, but maybe this helps? https://github.com/nihui/dain-ncnn-vulkan https://github.com/nihui/rife-ncnn-vulkan

I can recommend DAIN for the 60 fps: https://github.com/nihui/dain-ncnn-vulkan

link to the instructions https://github.com/nihui/dain-ncnn-vulkan/blob/master/README.md

For AMD. But it takes forever to even do a few seconds on AMD for me (5700 XT; and Nvidia GPUs are impossible to get..). And NVIDIA. Generally you split the video in single frames; interpolate them with the DAIN AI Models (Idk which model is best for animation) and the images that the model created are then re-rendered to a video with ffmpeg.

Main Code: https://github.com/nateraw/stable-diffusion-videos/

So I've continued to experiment with how many papers I can fit into a single pipe and have them play nicely together. The images below were created by combining the panorama code from omerbt/MultiDiffusion with the ideas from albarji/mixture-of-diffusers. Also turns out nateraw/stable-diffusion-videos can be seen as a special case of a panorama (in latent space rather than prompt space).

https://huggingface.co/spaces/tomg-group-umd/pez-dispenser https://huggingface.co/spaces/AIML-TUDA/safe-stable-diffusion https://huggingface.co/spaces/AIML-TUDA/semantic-diffusion https://github.com/nateraw/stable-diffusion-videos

The goal is to make a (short) video out of a given first and last frame. It is similar to what this guy does (https://github.com/nateraw/stable-diffusion-videos (7sec example video half way down page)). But instead of starting and ending with a prompt, I want to start and end with 2 different frames.

Hey Y'all! We've been working on a tool that extends Nate Raw's Stable Diffusion Videos repo and makes it as easy as possible to use for artists and are having a competition this week to stress test the beta and see who can use it to make the most compelling short video (40 seconds max)

Of course! There might be a better process but mainly used: 1.) Nate Raw's repo for morphing between prompts https://github.com/nateraw/stable-diffusion-videos 2.) Google FILM interpolation to smooth out transitions https://github.com/google-research/frame-interpolation

There are directions in the readme and an example script.

This does exactly that - https://github.com/nateraw/stable-diffusion-videos