deforum-stable-diffusion VS deforum-for-automatic1111-webui

This looks a fine-tune of the classic zero123 (https://github.com/cvlab-columbia/zero123) I’m excited to check out the quality improvements.

Though 3d model synthesis is one use case, I found the less advertised base reprojection model to be more useful for gamedev at the moment. You can generate a multiview spritesheet from an image, and it’s fast enough for synthesis during a gameplay session. I couldn’t get a good quality/time balance to do the same with the 3d models, and the lack of mesh rigging or animation combined with imperfections in a fully 3d model tends to break the suspension of disbelief compared to what players are used to. I’m this will change as the tech develops and we layer more AI on top (automatic animation synthesis is an active research area).

If you’re interested in this you might also want to check out deforum (https://github.com/deforum-art/deforum-stable-diffusion) which provides even more powerful camera controls on top of stable diffusion designed for full scenes rather than single objects.

2/Deforum – Open source and good at developing trippy videos – I have not used this too much (https://github.com/deforum-art/deforum-stable-diffusion)

Why is StabilityAI releasing what seems to be a clone of Deforum ?

might be deforum (project) (webui extension) or img2img loopback wave (webui extension). it's definitely something with img2img though. that's clear from how it's warping.

Sorry, I missed it, thanks to Deforum: https://github.com/deforum-art/deforum-stable-diffusion

Have you seen Deforum? Kinda sounds like you're about to reinvent it. :)

I'm talking about it https://github.com/deforum-art/deforum-stable-diffusion It is not possible to put in a separate environment.

Deforum https://github.com/deforum-art/deforum-for-automatic1111-webui

might be deforum (project) (webui extension) or img2img loopback wave (webui extension). it's definitely something with img2img though. that's clear from how it's warping.

Thanks for the workflow. To simplify the process I created a PR that hopefully will make it a bit faster, specially if you're working in rented cloud GPU's.

The sequence after the train with the flashing background is the only exception (1:03 to 1:24), that part was made using the Deforum notebook: https://deforum.github.io/

ive gotten better results applying some color correction to my loops using the same way as they do in deforum https://github.com/deforum-art/deforum-for-automatic1111-webui/blob/automatic1111-webui/scripts/deforum_helpers/colors.py

🔸 Deforum extension for Automatic1111 (Local Install): https://github.com/deforum-art/deforum-for-automatic1111-webui

Deforum plugin settings files: render, coloration. The black and white canny edges + scribble-mode mid-stage rendered video. You can check the prompts keyframing file to see if I got the characters names right. Despite my best efforts, some of the characters appearances were off, I think it was either the character names were incorrectly spelled (I was referring to the Touhou wiki) or they had been underrepresented in the dataset or there was overlap between their naming and some other franchises. There is also the issue of random unrelated characters appearing when the prompt contains no characters and instead is purely abstract, I guess it relates to this specific model retraining.