paint-with-words-sd
ComfyUI
paint-with-words-sd | ComfyUI | |
---|---|---|
13 | 125 | |
618 | 34,594 | |
- | - | |
5.2 | 9.9 | |
about 1 year ago | about 20 hours ago | |
Jupyter Notebook | Python | |
MIT License | GNU General Public License v3.0 only |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
paint-with-words-sd
-
paint with words with loras and multicontrolnet (will pay if needed)
I am refering to this btw: https://github.com/cloneofsimo/paint-with-words-sd
-
More control than ControlNet - code is out for MultiDiffusion Region Control, a prompt on each mask
This essentially supercharges the Nvidia eDiffi / SD paint-with-words attempts done for the same thing previously.
- "Segmentation" ControlNet preprocessor options
-
I figured out a way to apply different prompts to different sections of the image with regular Stable Diffusion models and it works pretty well.
There is stable diffusion paint with words GitHub which probably does exactly this, but no UI ever: https://github.com/cloneofsimo/paint-with-words-sd
-
What do you think will be added/created next?
personally i want to see the ediffi paint w/words stable diffusion extension https://github.com/cloneofsimo/paint-with-words-sd/commit/789419e3a34f43a1454df5a940020cfa531fbc63 that clonesofimo was working on before he stopped
- Will models have to be retrained for when this feature is eventually added into SD?
-
Paint with words (aka NVIDIA eDiff-I)
Just found there is a repo for an NVIDIA eDiff-I style img2img workflow for Stable Diffusion. For those unfamiliar, this lets you specify where parts of your text prompt should be placed in the image giving you much greater control on the composition e.g.
-
Different Models = Different prompts?
Paint-with-Words might eventually allow something along those lines, but it's a bit awkward to use now, and AFAIK you still get bleedthrough between multiple human subjects.
-
eDiff-I: A new Text-to-Image Diffusion Model with Ensemble of Expert Denoisers
someone attempted something like paint with words but I think Nvidia's version is better looking.
- Paint with words? What is next? Hope this gets to be a module in automatic 1111 soon.
ComfyUI
-
ComflowySpace: An open-source version of better ComfyUI
The non standard licensing puts me off in contributing or using this. It is frustrating how the phrase opensource has been diluted in the AI/ML community. ComfyUI has a GPL license [1] while this project uses this [2]. I honestly don't know where I stand since this is a legal document using non-standard phrasing to describe how the rights around the source code.
This is a project that uses a custom license with less rights provided than the ComfyUI project it self-describes as improving. Am not sure the title is reflective of the project.
[1] - https://github.com/comfyanonymous/ComfyUI/blob/master/LICENS...
-
Show HN: I made an app to use local AI as daily driver
* LLaVA model: I'll add more documentation. You are right Llava could not generate images. For image generation I don't have immediate plans, but checkout these projects for local image generation.
- https://diffusionbee.com/
- https://github.com/comfyanonymous/ComfyUI
- https://github.com/AUTOMATIC1111/stable-diffusion-webui
-
Show HN: ML Blocks – Deploy multimodal AI workflows without code
Check out ComfyUI for a much more advanced and open source version of this.
https://github.com/comfyanonymous/ComfyUI
-
Stable Code 3B: Coding on the Edge
I use Stable Diffusion family models for innovative art products.
On a small scale, you have to professionalize ComfyUI’s development. My PR to make it installable and to make a plugin ecosystem that makes sense should not be sitting unmerged (https://github.com/comfyanonymous/ComfyUI/pull/298).
On a medium scale, CLIP is holding you back. I would eagerly buy a 48GB card to accommodate a batch size 1, gradient checkpointed LoRA-trainable model with T5 for conditioning. I want PixArt-a or DeepFloyd/IF with the SDXL dataset and training. I get I can achieve so much with SDXL on 24GB, including just barely a fine tuning, I understand the engineering decisions here, but it’s too weak on prompts.
On a large scale, I’m willing to spend a little money up front. In those conditions you can be far more innovative, you don’t have to make everything for $0. Shane Carruth didn’t make Primer for $0. I’m sure you’ve seen this movie, you get how astoundingly good it is. But he still spent something. He spent only slightly more than an RTX 6000 Ada.
Innovators have budgets. It’s still worth releasing the most powerful possible model for expensive hardware, this is why everyone is talking about Mixtral, but it’s especially true of visual art.
-
Show HN: Comflowy – A ComfyUI Tutorial for Beginners
It's litegraph.js [1] and seems to be the only lib they include in /web [2] :
[1] https://github.com/jagenjo/litegraph.js
[2] https://github.com/comfyanonymous/ComfyUI/tree/master/web/li...
-
ComfyUI on Windows 7?
It's possibly you might get a later version of Comfy working, but I had no success with this method and the 1st Sept version of Comfy. The older versions are here under Assets... https://github.com/comfyanonymous/ComfyUI/releases
-
Seeking out an experienced and empathetic coding buddy.
That said, please do learn coding and don't get discouraged when somebody says to learn PyTorch or recommends using a Jupiter notebook with no further information on how to translate the skill into images. I would highly recommend some short term goals. Get your feet wet by taking apart the UIs. The comfy API documentation is here and the A1111 API documentation is here. There is a difference in completeness, welcome to programming. Writing nodes or plugins is also a good way to jump into this world. Custom wildcard logic might be very attractive to you if you aren't the type that want to deal with a nested file structure to simulate logic.
-
Need help installating ComfyUI
For example ComfyUI can simply be downloaded and run using the portable version (https://github.com/comfyanonymous/ComfyUI/releases/download/latest/ComfyUI_windows_portable_nvidia_cu121_or_cpu.7z) if your not comfortable using GIT etc.
-
Installing cumfyui manager on MacBook
https://github.com/comfyanonymous/ComfyUI scroll down to "Install"
- SAG (Self-Attention Guidance) for ComfyUI is here!
What are some alternatives?
openOutpaint - local offline javascript and html canvas outpainting gizmo for stable diffusion webUI API 🐠
stable-diffusion-webui - Stable Diffusion web UI
LECO - Low-rank adaptation for Erasing COncepts from diffusion models.
stable-diffusion-ui - Easiest 1-click way to install and use Stable Diffusion on your computer. Provides a browser UI for generating images from text prompts and images. Just enter your text prompt, and see the generated image. [Moved to: https://github.com/easydiffusion/easydiffusion]
Rerender_A_Video - [SIGGRAPH Asia 2023] Rerender A Video: Zero-Shot Text-Guided Video-to-Video Translation
InvokeAI - InvokeAI is a leading creative engine for Stable Diffusion models, empowering professionals, artists, and enthusiasts to generate and create visual media using the latest AI-driven technologies. The solution offers an industry leading WebUI, supports terminal use through a CLI, and serves as the foundation for multiple commercial products.
openOutpaint-webUI-extension - direct A1111 webUI extension for openOutpaint
sd-webui-controlnet - WebUI extension for ControlNet
daam - Diffusion attentive attribution maps for interpreting Stable Diffusion.
stable-diffusion-webui-two-shot - Latent Couple extension (two shot diffusion port)
a1111-nevysha-comfy-ui - A collection of tweak to improve Auto1111 UI//UX [Moved to: https://github.com/Nevysha/Cozy-Nest]