stablediffusion VS ComfyUI

With this tutorial's help, you can generate images with AI on your own computer with Stable Diffusion.

If your PC has a GPU(Nvidia RTX 30series+ recommended) of VRAM more than 4GB then try training your own Stable Diffusion model.

Command: "git" clone "https://github.com/Stability-AI/stablediffusion.git" "C:\Users\Naveed\Documents\A1111 Web UI Autoinstaller\stable-diffusion-webui\repositories\stable-diffusion-stability-ai"

Automatic11112 and sygil-webui aren't "distributions" of Stable Diffusion. This is a repository with some distributions of Stable Diffusion.

New stable diffusion finetune (Stable unCLIP 2.1, Hugging Face) at 768x768 resolution, based on SD2.1-768. This model allows for image variations and mixing operations as described in Hierarchical Text-Conditional Image Generation with CLIP Latents, and, thanks to its modularity, can be combined with other models such as KARLO. Comes in two variants: Stable unCLIP-L and Stable unCLIP-H, which are conditioned on CLIP ViT-L and ViT-H image embeddings, respectively. Instructions are available here.

This model will soon be open-sourced in StabilityAI’s GitHub.

Another question, if that's ok? Stable Diffusion 2.0 - https://github.com/Stability-AI/stablediffusion - if I wanted to use that, do I follow along their instructions and it will work on the M1 still, or you advise against it?

Stable Diffusion (2D Image Generation and Animation) https://github.com/CompVis/stable-diffusion (Stable Diffusion V1) https://huggingface.co/CompVis/stable-diffusion (Stable Diffusion Checkpoints 1.1-1.4) https://huggingface.co/runwayml/stable-diffusion-v1-5 (Stable Diffusion Checkpoint 1.5) https://github.com/Stability-AI/stablediffusion (Stable Difusion V2) https://huggingface.co/stabilityai/stable-diffusion-2-1/tree/main (Stable Diffusion Checkpoint 2.1) Stable Diffusion Automatic 1111 Webui and Extensions https://github.com/AUTOMATIC1111/stable-diffusion-webui (WebUI - Easier to use) PLEASE NOTE, MANY EXTENSIONS CAN BE INSTALLED FROM THE WEBUI BY CLICK "AVAILABLE" OR "INSTALL FROM URL" BUT YOU MAY STILL NEED TO DOWNLOAD THE MODEL CHECKPOINTS! https://github.com/Mikubill/sd-webui-controlnet (Control Net Extension - Use various models to control your image generation, useful for animation and temporal consistency) https://huggingface.co/lllyasviel/ControlNet/tree/main/models (Control Net Checkpoints -Canny, Normal, OpenPose, Depth, ect.) https://github.com/thygate/stable-diffusion-webui-depthmap-script (Depth Map Extension - Generate high-resolution depthmaps and animated videos or export to 3d modeling programs) https://github.com/graemeniedermayer/stable-diffusion-webui-normalmap-script (Normal Map Extension - Generate high-resolution normal maps for use in 3d programs) https://github.com/d8ahazard/sd_dreambooth_extension (Dream Booth Extension - Train your own objects, people, or styles into Stable Diffusion) https://github.com/deforum-art/sd-webui-deforum (Deforum - Generate Weird 2D animations) https://github.com/deforum-art/sd-webui-text2video (Deforum Text2Video - Generate videos from texts prompts using ModelScope or VideoCrafter)

Stable Diffusion's code : https://github.com/Stability-AI/stablediffusion

But if you want to run a 2.x-based model, you'll need to download the corresponding YAML file (either the standard one – v2-inference-v.yaml – from Github or the one that is distributed with the model, if it requires a special one), rename it to have the same name as the model, and place it in the models folder alongside the model.

The non standard licensing puts me off in contributing or using this. It is frustrating how the phrase opensource has been diluted in the AI/ML community. ComfyUI has a GPL license [1] while this project uses this [2]. I honestly don't know where I stand since this is a legal document using non-standard phrasing to describe how the rights around the source code.

This is a project that uses a custom license with less rights provided than the ComfyUI project it self-describes as improving. Am not sure the title is reflective of the project.

[1] - https://github.com/comfyanonymous/ComfyUI/blob/master/LICENS...

* LLaVA model: I'll add more documentation. You are right Llava could not generate images. For image generation I don't have immediate plans, but checkout these projects for local image generation.

- https://diffusionbee.com/

- https://github.com/comfyanonymous/ComfyUI

- https://github.com/AUTOMATIC1111/stable-diffusion-webui

Check out ComfyUI for a much more advanced and open source version of this.

https://github.com/comfyanonymous/ComfyUI

I use Stable Diffusion family models for innovative art products.

On a small scale, you have to professionalize ComfyUI’s development. My PR to make it installable and to make a plugin ecosystem that makes sense should not be sitting unmerged (https://github.com/comfyanonymous/ComfyUI/pull/298).

On a medium scale, CLIP is holding you back. I would eagerly buy a 48GB card to accommodate a batch size 1, gradient checkpointed LoRA-trainable model with T5 for conditioning. I want PixArt-a or DeepFloyd/IF with the SDXL dataset and training. I get I can achieve so much with SDXL on 24GB, including just barely a fine tuning, I understand the engineering decisions here, but it’s too weak on prompts.

On a large scale, I’m willing to spend a little money up front. In those conditions you can be far more innovative, you don’t have to make everything for $0. Shane Carruth didn’t make Primer for $0. I’m sure you’ve seen this movie, you get how astoundingly good it is. But he still spent something. He spent only slightly more than an RTX 6000 Ada.

Innovators have budgets. It’s still worth releasing the most powerful possible model for expensive hardware, this is why everyone is talking about Mixtral, but it’s especially true of visual art.

It's litegraph.js [1] and seems to be the only lib they include in /web [2] :

[1] https://github.com/jagenjo/litegraph.js

[2] https://github.com/comfyanonymous/ComfyUI/tree/master/web/li...

It's possibly you might get a later version of Comfy working, but I had no success with this method and the 1st Sept version of Comfy. The older versions are here under Assets... https://github.com/comfyanonymous/ComfyUI/releases

That said, please do learn coding and don't get discouraged when somebody says to learn PyTorch or recommends using a Jupiter notebook with no further information on how to translate the skill into images. I would highly recommend some short term goals. Get your feet wet by taking apart the UIs. The comfy API documentation is here and the A1111 API documentation is here. There is a difference in completeness, welcome to programming. Writing nodes or plugins is also a good way to jump into this world. Custom wildcard logic might be very attractive to you if you aren't the type that want to deal with a nested file structure to simulate logic.

For example ComfyUI can simply be downloaded and run using the portable version (https://github.com/comfyanonymous/ComfyUI/releases/download/latest/ComfyUI_windows_portable_nvidia_cu121_or_cpu.7z) if your not comfortable using GIT etc.

https://github.com/comfyanonymous/ComfyUI scroll down to "Install"