deforum-for-automatic1111-webui VS ComfyUI_ADV_CLIP_emb

Deforum https://github.com/deforum-art/deforum-for-automatic1111-webui

might be deforum (project) (webui extension) or img2img loopback wave (webui extension). it's definitely something with img2img though. that's clear from how it's warping.

Thanks for the workflow. To simplify the process I created a PR that hopefully will make it a bit faster, specially if you're working in rented cloud GPU's.

The sequence after the train with the flashing background is the only exception (1:03 to 1:24), that part was made using the Deforum notebook: https://deforum.github.io/

ive gotten better results applying some color correction to my loops using the same way as they do in deforum https://github.com/deforum-art/deforum-for-automatic1111-webui/blob/automatic1111-webui/scripts/deforum_helpers/colors.py

🔸 Deforum extension for Automatic1111 (Local Install): https://github.com/deforum-art/deforum-for-automatic1111-webui

Deforum plugin settings files: render, coloration. The black and white canny edges + scribble-mode mid-stage rendered video. You can check the prompts keyframing file to see if I got the characters names right. Despite my best efforts, some of the characters appearances were off, I think it was either the character names were incorrectly spelled (I was referring to the Touhou wiki) or they had been underrepresented in the dataset or there was overlap between their naming and some other franchises. There is also the issue of random unrelated characters appearing when the prompt contains no characters and instead is purely abstract, I guess it relates to this specific model retraining.

Advanced CLIP Text Encode

With SDXL on the horizon, I've gone ahead and updated my prompt weighting nodes for ComfyUI and did some quick testing.

There are different ways of interpreting the up or down-weighting of words in prompts. A1111 for instance simply scales the associated vector by the prompt weight, while ComfyUI by default calculates a travel direction from the prompt and an empty prompt. This results in markedly different behavior at higher weighting. I made a node for ComfyUI that lets you pick and mix different ways of interpreting these token weights, so that people can play around with and explore these things 😁. You can find the github page with a more in depth explanation here