paint-with-words-sd
LECO
paint-with-words-sd | LECO | |
---|---|---|
13 | 1 | |
618 | 289 | |
- | - | |
5.2 | 7.8 | |
about 1 year ago | 4 months ago | |
Jupyter Notebook | Jupyter Notebook | |
MIT License | Apache License 2.0 |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
paint-with-words-sd
-
paint with words with loras and multicontrolnet (will pay if needed)
I am refering to this btw: https://github.com/cloneofsimo/paint-with-words-sd
-
More control than ControlNet - code is out for MultiDiffusion Region Control, a prompt on each mask
This essentially supercharges the Nvidia eDiffi / SD paint-with-words attempts done for the same thing previously.
- "Segmentation" ControlNet preprocessor options
-
I figured out a way to apply different prompts to different sections of the image with regular Stable Diffusion models and it works pretty well.
There is stable diffusion paint with words GitHub which probably does exactly this, but no UI ever: https://github.com/cloneofsimo/paint-with-words-sd
-
What do you think will be added/created next?
personally i want to see the ediffi paint w/words stable diffusion extension https://github.com/cloneofsimo/paint-with-words-sd/commit/789419e3a34f43a1454df5a940020cfa531fbc63 that clonesofimo was working on before he stopped
- Will models have to be retrained for when this feature is eventually added into SD?
-
Paint with words (aka NVIDIA eDiff-I)
Just found there is a repo for an NVIDIA eDiff-I style img2img workflow for Stable Diffusion. For those unfamiliar, this lets you specify where parts of your text prompt should be placed in the image giving you much greater control on the composition e.g.
-
Different Models = Different prompts?
Paint-with-Words might eventually allow something along those lines, but it's a bit awkward to use now, and AFAIK you still get bleedthrough between multiple human subjects.
-
eDiff-I: A new Text-to-Image Diffusion Model with Ensemble of Expert Denoisers
someone attempted something like paint with words but I think Nvidia's version is better looking.
- Paint with words? What is next? Hope this gets to be a module in automatic 1111 soon.
LECO
-
Unified Concept Editing in Diffusion Models
Editing models in seconds. This is an upgrade to the lora sliders (https://erasing.baulab.info and https://github.com/p1atdev/LECO) but faster training with no damage to the model prior knowledge! Check out their code: https://github.com/rohitgandikota/unified-concept-editing
What are some alternatives?
ComfyUI - The most powerful and modular stable diffusion GUI, api and backend with a graph/nodes interface.
daam - Diffusion attentive attribution maps for interpreting Stable Diffusion.
openOutpaint - local offline javascript and html canvas outpainting gizmo for stable diffusion webUI API 🐠
erasing - Erasing Concepts from Diffusion Models
Rerender_A_Video - [SIGGRAPH Asia 2023] Rerender A Video: Zero-Shot Text-Guided Video-to-Video Translation
lora - Using Low-rank adaptation to quickly fine-tune diffusion models.
openOutpaint-webUI-extension - direct A1111 webUI extension for openOutpaint
infinite-zoom-stable-diffusion - resources for creating Ininite zoom video using Stable Diffiusion, you can use multiple prompts and it is easy to use.
unified-concept-editing - Unified Concept Editing in Diffusion Models
stable-diffusion-webui-two-shot - Latent Couple extension (two shot diffusion port)
LCM-LoRA - LCM LoRA