kohya_ss VS ComfyUI

Many of the options are explained here https://github.com/bmaltais/kohya_ss/wiki/LoRA-training-parameters

training in BF16 might solve this issue from what I saw in this ticket. I know other people ran into the issue too https://github.com/bmaltais/kohya_ss/issues/1382

for lycoris loras you can use the command-line script from the kohya-ss repo https://github.com/bmaltais/kohya_ss/blob/master/networks/merge_lora.py i have an older version checked out from late july, it had a separate merge_lycoris.py for for this purpose, it's probably unified now in a single file

I usually do that with kohya_ss, a tool made for making LoRAs and finetunes. It might be a bit of a pain to set up just to do this one task, but if nobody gives you an easier method, look into it. https://github.com/bmaltais/kohya_ss

After that, make your staging directory, and do the git clone https://github.com/bmaltais/kohya_ss.git, and navigate inside of it. Now, here's where things can become a pain. I used pyenv to set my system level python to 3.10.6 with pyenv global 3.10.6, though you can probably just use "local" and do it for the current shell. You NEED it to be active however before you set up your venv. If you do python --version and get 3.10.6, you're ready for this next part. Make your venv with python -m venv venv. This is the simplest way, it'll create a virtual environment in your current folder named venv. You'll do a source venv/bin/activate and then do which python to make sure it's using the python from the venv. Now for the fun part. The included setup scripts have been flaky for me, so I just went through the requirements and installed everything by hand. I'm going to do this guide right now for nvidia, because I just got a 4090 for this stuff. If this ends up working well for others and there's demand, I'll try to reproduce this for AMD (But I'll be honest, I got an nvidia card because bitsandbytes doesn't have full rocm support, nor do most libraries, so it's not very reliable). After installing everything and testing it works at least at a basic level for dreambooth training, my finished requirements.txt for pip is as below:

Earlier I created a post where I asked for recommendations for LoRA model training tutorials. The first one I looked at used the kohya_ss GUI. That GitHub repo already has two tutorials, which are quite good, so I ended up using those:

I have tried my best to research this issue and have not come up with much. It is obvious that its a backend issue right? The guides that I used https://github.com/bmaltais/kohya_ss and https://github.com/pyenv-win/pyenv-win/

I am training on a 12GB 3060 using kohya_ss. Is there a setting or something I'm missing?

The non standard licensing puts me off in contributing or using this. It is frustrating how the phrase opensource has been diluted in the AI/ML community. ComfyUI has a GPL license [1] while this project uses this [2]. I honestly don't know where I stand since this is a legal document using non-standard phrasing to describe how the rights around the source code.

This is a project that uses a custom license with less rights provided than the ComfyUI project it self-describes as improving. Am not sure the title is reflective of the project.

[1] - https://github.com/comfyanonymous/ComfyUI/blob/master/LICENS...

* LLaVA model: I'll add more documentation. You are right Llava could not generate images. For image generation I don't have immediate plans, but checkout these projects for local image generation.

- https://diffusionbee.com/

- https://github.com/comfyanonymous/ComfyUI

- https://github.com/AUTOMATIC1111/stable-diffusion-webui

Check out ComfyUI for a much more advanced and open source version of this.

https://github.com/comfyanonymous/ComfyUI

I use Stable Diffusion family models for innovative art products.

On a small scale, you have to professionalize ComfyUI’s development. My PR to make it installable and to make a plugin ecosystem that makes sense should not be sitting unmerged (https://github.com/comfyanonymous/ComfyUI/pull/298).

On a medium scale, CLIP is holding you back. I would eagerly buy a 48GB card to accommodate a batch size 1, gradient checkpointed LoRA-trainable model with T5 for conditioning. I want PixArt-a or DeepFloyd/IF with the SDXL dataset and training. I get I can achieve so much with SDXL on 24GB, including just barely a fine tuning, I understand the engineering decisions here, but it’s too weak on prompts.

On a large scale, I’m willing to spend a little money up front. In those conditions you can be far more innovative, you don’t have to make everything for $0. Shane Carruth didn’t make Primer for $0. I’m sure you’ve seen this movie, you get how astoundingly good it is. But he still spent something. He spent only slightly more than an RTX 6000 Ada.

Innovators have budgets. It’s still worth releasing the most powerful possible model for expensive hardware, this is why everyone is talking about Mixtral, but it’s especially true of visual art.

It's litegraph.js [1] and seems to be the only lib they include in /web [2] :

[1] https://github.com/jagenjo/litegraph.js

[2] https://github.com/comfyanonymous/ComfyUI/tree/master/web/li...

It's possibly you might get a later version of Comfy working, but I had no success with this method and the 1st Sept version of Comfy. The older versions are here under Assets... https://github.com/comfyanonymous/ComfyUI/releases

That said, please do learn coding and don't get discouraged when somebody says to learn PyTorch or recommends using a Jupiter notebook with no further information on how to translate the skill into images. I would highly recommend some short term goals. Get your feet wet by taking apart the UIs. The comfy API documentation is here and the A1111 API documentation is here. There is a difference in completeness, welcome to programming. Writing nodes or plugins is also a good way to jump into this world. Custom wildcard logic might be very attractive to you if you aren't the type that want to deal with a nested file structure to simulate logic.

For example ComfyUI can simply be downloaded and run using the portable version (https://github.com/comfyanonymous/ComfyUI/releases/download/latest/ComfyUI_windows_portable_nvidia_cu121_or_cpu.7z) if your not comfortable using GIT etc.

https://github.com/comfyanonymous/ComfyUI scroll down to "Install"