ComfyUI VS Fooocus

The non standard licensing puts me off in contributing or using this. It is frustrating how the phrase opensource has been diluted in the AI/ML community. ComfyUI has a GPL license [1] while this project uses this [2]. I honestly don't know where I stand since this is a legal document using non-standard phrasing to describe how the rights around the source code.

This is a project that uses a custom license with less rights provided than the ComfyUI project it self-describes as improving. Am not sure the title is reflective of the project.

[1] - https://github.com/comfyanonymous/ComfyUI/blob/master/LICENS...

* LLaVA model: I'll add more documentation. You are right Llava could not generate images. For image generation I don't have immediate plans, but checkout these projects for local image generation.

- https://diffusionbee.com/

- https://github.com/comfyanonymous/ComfyUI

- https://github.com/AUTOMATIC1111/stable-diffusion-webui

Check out ComfyUI for a much more advanced and open source version of this.

https://github.com/comfyanonymous/ComfyUI

I use Stable Diffusion family models for innovative art products.

On a small scale, you have to professionalize ComfyUI’s development. My PR to make it installable and to make a plugin ecosystem that makes sense should not be sitting unmerged (https://github.com/comfyanonymous/ComfyUI/pull/298).

On a medium scale, CLIP is holding you back. I would eagerly buy a 48GB card to accommodate a batch size 1, gradient checkpointed LoRA-trainable model with T5 for conditioning. I want PixArt-a or DeepFloyd/IF with the SDXL dataset and training. I get I can achieve so much with SDXL on 24GB, including just barely a fine tuning, I understand the engineering decisions here, but it’s too weak on prompts.

On a large scale, I’m willing to spend a little money up front. In those conditions you can be far more innovative, you don’t have to make everything for $0. Shane Carruth didn’t make Primer for $0. I’m sure you’ve seen this movie, you get how astoundingly good it is. But he still spent something. He spent only slightly more than an RTX 6000 Ada.

Innovators have budgets. It’s still worth releasing the most powerful possible model for expensive hardware, this is why everyone is talking about Mixtral, but it’s especially true of visual art.

It's litegraph.js [1] and seems to be the only lib they include in /web [2] :

[1] https://github.com/jagenjo/litegraph.js

[2] https://github.com/comfyanonymous/ComfyUI/tree/master/web/li...

It's possibly you might get a later version of Comfy working, but I had no success with this method and the 1st Sept version of Comfy. The older versions are here under Assets... https://github.com/comfyanonymous/ComfyUI/releases

That said, please do learn coding and don't get discouraged when somebody says to learn PyTorch or recommends using a Jupiter notebook with no further information on how to translate the skill into images. I would highly recommend some short term goals. Get your feet wet by taking apart the UIs. The comfy API documentation is here and the A1111 API documentation is here. There is a difference in completeness, welcome to programming. Writing nodes or plugins is also a good way to jump into this world. Custom wildcard logic might be very attractive to you if you aren't the type that want to deal with a nested file structure to simulate logic.

For example ComfyUI can simply be downloaded and run using the portable version (https://github.com/comfyanonymous/ComfyUI/releases/download/latest/ComfyUI_windows_portable_nvidia_cu121_or_cpu.7z) if your not comfortable using GIT etc.

https://github.com/comfyanonymous/ComfyUI scroll down to "Install"

But we can come to a pretty realistic (although not as accurate) conclusion if we put our minds to it. I chose Fooocus for this example, which is the most straightforward (and I believe popular) stable diffusion GUI out there. Let's start simple:

# Install the JuiceFS client. !curl -sSL https://d.juicefs.com/install | sh - # Mount the JuiceFS file system. !juicefs mount rediss://:[email protected]/1 myjfs -d # Create the directory structure for Fooocus models in JuiceFS. !mkdir -p myjfs/models/{checkpoints,loras,embeddings,vae_approx,upscale_models,inpaint,controlnet,clip_vision} # Clone the Fooocus repository. !git clone https://github.com/lllyasviel/Fooocus.git

That looks very impressive unless the demo is cherrypicked, would be great if this could be implemented into a frontend like Fooocus https://github.com/lllyasviel/Fooocus

You might be thinking of Fooocus: https://github.com/lllyasviel/Fooocus

The Stable Diffusion web interface that got a lot of people's attention originally was Automatic1111: https://github.com/AUTOMATIC1111/stable-diffusion-webui

Fooocus is definitely more beginner friendly. It does a lot of the prompt engineering for you. Automatic1111 has a ton of plugins, most notably ControlNet which gives you fine grained control over the images, but there is a learning curve.

I just installed this last night on my laptop:

https://github.com/lllyasviel/Fooocus

Highly recommend:

>"Looking up from the deck of golden gate bridge at the towers and metal work, the towers rise and arch back in an ominous and foreboding manner. more artistic, like an alphonse mucha propaganda poster - slightly fish-eye feeling" -- https://i.imgur.com/vyNg79f.jpg

the local UI and 1.27.0.0.1 - https://i.imgur.com/wRwghuN.jpg

Why not use the Google Colab notebook while it's still a free option at: https://github.com/lllyasviel/Fooocus It's not bad. I've been using the Colab Fooocus notebook and A1111 on Sage with the 4 free hours of GPU time. The Colab has the Juggernaut model preloaded, but I combined some code from another notebook to add other models and loras.

i'm trying to test the abilities of image generators and the risks that can comes with it. and i'm looking for an image generator that work locally and has no limits. i used the Fooocus project from github and the juggernut model and it's capable of generating nude pictures but not fully nude pictures. and it doesn't work will with bloody scenes. any recommendation for a better model

I recently saw this video from Fireship and I started wondering about licensing of SD models and frameworks. Fireship shows Fooocus and advertises it as a cool solution. What I started wondering about is, Fooocus downloads a couple of models: Juggernaut XL, some control nets, some loras. What licensing is tied to all of this? One I am most insterested in is JuggernautXL, on civitai it's listed as having CreativeML Open RAIL++-M license but in the description there's remark: For business inquires, commercial licensing, custom models, and consultation contact me under [email protected] There's a lot of separate parts going on in AI frameworks and it's a bit unclear to understand if I can use this commercialy.

For running locally w/o a lot of "hazzle" i would recommend using https://github.com/lllyasviel/Fooocus and the "Sticker" style, which is available within the UI (Advanced tab). If you need lot of text directly in you images, you may have difficulties with SD and other AI models. In this case LoRas could help. Example: https://civitai.com/posts/880523 (check details for used prompt) In this case i used the following LoRa (a LoRa is kind of specialized submodel for style, concept or person) .