sd-pixel VS sd-webui-deforum

sd-pixelalmost perfect, but! I can't change quantization method, dithering and use a custom palette.

I’m thinking about constructing a file based on this default: https://github.com/deforum-art/sd-webui-deforum/blob/automatic1111-webui/scripts/default_settings.txt

I asked Audiocraft to make me a "chill hip hop beat", I used framesync.xyz to make keyframes for A1111 Deforum extension. Unfortunately, I don't have the settings file anymore, but it was pretty much just a 26s clip at 15fps (440 frames) with a single prompt "a surreal painting by Magritte" and the usual negative prompt magic voodoo. Then, for every clip I used the last frame of the previous clip as init frame. I render at 512x512 and then use ESRGAN4x to upscale to 2048x2048

https://github.com/deforum-art/sd-webui-deforum - for video generation

This kind of animation can be done with Deforum, a rather advanced add-on for Stable Diffusion. Be aware thought that you will have to really work yourself into that one. Also as a heads up, you probably won't be able to just use existing lyrics as prompts, rather create your different prompts based on the lyrics.

The tools you need for that are Stable DIffusion (e.g., with the Automatic1111 UI), therein you install Deforum. The github pages for both outline the installation process. Here's also a decent tutorial to get you started with Deforum, as it may seem a bit complex at first: Link

There’s also video to video with deforum, and text to video with deforum and modelscope. I am not really that familiar with this as I only started recently, so don’t blindly believe this.

Created with SD 1.5, Automatic1111 UI, Deforum plugin. This was attempt number 10 or so of this prompt, with lots of small settings tweaks between.