paint-with-words-sd
txt2mask
paint-with-words-sd | txt2mask | |
---|---|---|
13 | 24 | |
618 | 507 | |
- | - | |
5.2 | 2.6 | |
about 1 year ago | over 1 year ago | |
Jupyter Notebook | Python | |
MIT License | - |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
paint-with-words-sd
-
paint with words with loras and multicontrolnet (will pay if needed)
I am refering to this btw: https://github.com/cloneofsimo/paint-with-words-sd
-
More control than ControlNet - code is out for MultiDiffusion Region Control, a prompt on each mask
This essentially supercharges the Nvidia eDiffi / SD paint-with-words attempts done for the same thing previously.
- "Segmentation" ControlNet preprocessor options
-
I figured out a way to apply different prompts to different sections of the image with regular Stable Diffusion models and it works pretty well.
There is stable diffusion paint with words GitHub which probably does exactly this, but no UI ever: https://github.com/cloneofsimo/paint-with-words-sd
-
What do you think will be added/created next?
personally i want to see the ediffi paint w/words stable diffusion extension https://github.com/cloneofsimo/paint-with-words-sd/commit/789419e3a34f43a1454df5a940020cfa531fbc63 that clonesofimo was working on before he stopped
- Will models have to be retrained for when this feature is eventually added into SD?
-
Paint with words (aka NVIDIA eDiff-I)
Just found there is a repo for an NVIDIA eDiff-I style img2img workflow for Stable Diffusion. For those unfamiliar, this lets you specify where parts of your text prompt should be placed in the image giving you much greater control on the composition e.g.
-
Different Models = Different prompts?
Paint-with-Words might eventually allow something along those lines, but it's a bit awkward to use now, and AFAIK you still get bleedthrough between multiple human subjects.
-
eDiff-I: A new Text-to-Image Diffusion Model with Ensemble of Expert Denoisers
someone attempted something like paint with words but I think Nvidia's version is better looking.
- Paint with words? What is next? Hope this gets to be a module in automatic 1111 soon.
txt2mask
-
Unpromted text2mask
Honestly I'd suggest just downloading the script instead, it's much easier to use and gives you boxes for the prompts rather than having to use all this stuff --> [txt2mask background[/txt2mask] It's up to you of course, but for me the extension conflicts with my favorite extension Dynamic Prompts anyway, so it had to go :( The stand-alone script still works mint tho haha.
-
Another Instruct Pix2Pix on video experiment: "Make it a bronze sculpture"
Can pix2pix be used with the txt2mask extension to easily isolate the dancer?
-
Any random hair colour function for automatic1111?
That looks similar to Unprompted, which is the successor to txt2mask which is probably much easier to use than the others (at least after a quick glance at the documentation).
-
In-painting Mask generation via API
automatic1111's webui now has a txt2mask script for inpainting. see here. Works great.
-
Will models have to be retrained for when this feature is eventually added into SD?
Separating colours into masks then rendering one at a time would just take a plugin, not a model update. It would be like this plugin only a lot easier as the objects are coming pre-masked by the colour instead of having to run recognition on them from the description https://github.com/ThereforeGames/txt2mask
-
[Inpainting] [Q] Want to remove a person/ group of people from an image.
It's pretty awesome but the developer recently said he's wrapping it into another extension of his and will not be updating the standalone script anymore. I'm sad about that because it works differently in the new extension and I find it a lot less convenient now, altho it's possible it will be changed again so I'm not giving up hope yet lol. Here's a link to the original version if you want to try it https://github.com/ThereforeGames/txt2mask
-
InstructPix2Pix - Stable Diffusion Combined With GPT-3 to "make it so"
There's a version of this idea already in the Automatic1111 distro, a script called 'txt2mask' that's on github here: https://github.com/ThereforeGames/txt2mask
-
Is it possible to replace object in image with object from another image
txt2mask
- Stable Diffusion links from around September 17, 2022 that I collected for further processing
-
Inpainting every frame using AE + SD
Perhaps you could txt2mask (https://github.com/ThereforeGames/txt2mask) to automate - i.e. you just need to have the text "fire hydrant" as your mask.
What are some alternatives?
ComfyUI - The most powerful and modular stable diffusion GUI, api and backend with a graph/nodes interface.
sd-dynamic-prompts - A custom script for AUTOMATIC1111/stable-diffusion-webui to implement a tiny template language for random prompt generation
openOutpaint - local offline javascript and html canvas outpainting gizmo for stable diffusion webUI API 🐠
stable-diffusion-prompt-inpainting - This project helps you do prompt-based inpainting without having to paint the mask - using Stable Diffusion and Clipseg
LECO - Low-rank adaptation for Erasing COncepts from diffusion models.
ECCV2022-RIFE - ECCV2022 - Real-Time Intermediate Flow Estimation for Video Frame Interpolation
Rerender_A_Video - [SIGGRAPH Asia 2023] Rerender A Video: Zero-Shot Text-Guided Video-to-Video Translation
gif2gif - Automatic1111 Animated Image (input/output) Extension
openOutpaint-webUI-extension - direct A1111 webUI extension for openOutpaint
stable-diffusion-webui - Stable Diffusion web UI
daam - Diffusion attentive attribution maps for interpreting Stable Diffusion.
clipseg - This repository contains the code of the CVPR 2022 paper "Image Segmentation Using Text and Image Prompts".