sd_lite VS Concurrent-gif2gif

The full details of how to do this are available on Github: latent jitter · thekitchenscientist/sd_lite but I will explain the idea briefly here. I have read this could be done with perlin or simplex noise but the code was too complex for my taste. This gets the job done with only minor modifications to the standard pipe.

I did successfully swap the effiel tower for the burj Khalifa but that required additional steps https://github.com/thekitchenscientist/sd_lite/wiki/latent-jitter

I use it in my pipe as a general image beautifier. https://github.com/thekitchenscientist/sd_lite/wiki/safe-latent-diffusion

The pipe is available at sd_lite/pipeline_stable_diffusion_multi.py (github.com) it combines:

This community has published many XY plots of CFG versus steps. https://github.com/thekitchenscientist/sd_lite/wiki/recommended The consistent theme is low CFG, lower steps; high CFG, more steps. UniPC can reach convergence in as few as 8 steps, so I increased by 1/3 to account for more complex prompts needing longer

the pipeline is available from github.com and is called in the usual way. The Technique requires the DDIM scheduler.

https://github.com/thekitchenscientist/sd_lite is where the code is. Version 1 of the multi-pipe is limited to images 512 high or wide but any number on the other dimension

Not to be deterred I hacked together some code to blend it all back together after the VAE but before the final colour balance. The pipe code is available on github. thekitchenscientist/sd_lite

I've been tinkering with different approaches to coherent animation, and one idea I can't seem to shake is rendering multiple frames from the same latent space, then resequencing them. I've had some success, but one major limitation is the scale of the initial noise/image. I can't generate against 16000x9000 worth of stitched frames.

What a cool idea. I've been pursuing something similar, in terms of maintaining subject consistency between "panels," but for animation frames. I wish I knew more about the diffusion pipeline to wrangle the "rendered from same latent space" benefit.