stable-diffusion-webui-tensorrt VS sd-webui-deforum

Also to note there is now the TensorRT to double your speed with. I have also calculated a few optimal settings for this as well. Max batch size of 7 with max height and width at 512. so 7x512x512. I have added a pull request for some gui changes and calculations for the extension which I am not sure have been added yet, but if you would like to have the gui tell you which settings for the max width, height, batch size, I can give you the code to update the extension. Also we added the ability to batch convert files from models>onnx and onnx>trt. Trts are the optimized files/models that are used to generate images 2+ times faster.

Unless something has changed in the last 7 days or for some reason the official repo isn't up to date, Loras are currently not supported and would have to be baked into the model to work.

I just followed the instructions here: https://github.com/AUTOMATIC1111/stable-diffusion-webui-tensorrt

I’m thinking about constructing a file based on this default: https://github.com/deforum-art/sd-webui-deforum/blob/automatic1111-webui/scripts/default_settings.txt

I asked Audiocraft to make me a "chill hip hop beat", I used framesync.xyz to make keyframes for A1111 Deforum extension. Unfortunately, I don't have the settings file anymore, but it was pretty much just a 26s clip at 15fps (440 frames) with a single prompt "a surreal painting by Magritte" and the usual negative prompt magic voodoo. Then, for every clip I used the last frame of the previous clip as init frame. I render at 512x512 and then use ESRGAN4x to upscale to 2048x2048

https://github.com/deforum-art/sd-webui-deforum - for video generation

This kind of animation can be done with Deforum, a rather advanced add-on for Stable Diffusion. Be aware thought that you will have to really work yourself into that one. Also as a heads up, you probably won't be able to just use existing lyrics as prompts, rather create your different prompts based on the lyrics.

The tools you need for that are Stable DIffusion (e.g., with the Automatic1111 UI), therein you install Deforum. The github pages for both outline the installation process. Here's also a decent tutorial to get you started with Deforum, as it may seem a bit complex at first: Link

There’s also video to video with deforum, and text to video with deforum and modelscope. I am not really that familiar with this as I only started recently, so don’t blindly believe this.

Created with SD 1.5, Automatic1111 UI, Deforum plugin. This was attempt number 10 or so of this prompt, with lots of small settings tweaks between.