LyCORIS VS bitsandbytes-rocm

Lycoris is another implementation of LoRA done by KohakuBlueleaf: https://github.com/KohakuBlueleaf/LyCORIS

Most guides focus on LoRa training as that has been around for longer. But I think LoHa can give better results. But the training is about half as fas it/s and it requires different training settings.

I've read in some tutorials that it is best that the value should be 64 or below, also here they suggest to not go over 64 ( https://github.com/KohakuBlueleaf/LyCORIS )

Does anyone know how to make LyCORIS models (https://github.com/KohakuBlueleaf/LyCORIS) work with inpainting models?

I think it was linked already but this is also relevant for LoRa: https://github.com/KohakuBlueleaf/LyCORIS Nice work!

There are some WIP evolutions of SD Lora in the works, like locon and lycoris.

https://github.com/KohakuBlueleaf/LyCORIS

You might want to expand LoRA to include LoCon and LoHa, (and also add a column for VRAM requirements) (Think of it as a more complete LoRA that works for the kernels in the convolutional units rather than just the weights for the feed-forward network), support is still quite limited, but it's starting to pick up steam https://github.com/KohakuBlueleaf/LyCORIS

Install dependencies like hipblas-devel hipsparse-devel hipcub-devel git python3.10 make libstdc++-devel accelerate, rocm and hip git clone https://github.com/bmaltais/kohya_ss && cd kohya_ss python3.10 -m venv venv source venv/bin/activate pip3 install torch==1.13.1 torchvision==0.14.1 torchtext==0.14.1 torchaudio==0.13.1 --index-url https://download.pytorch.org/whl/rocm5.2 # problems on 2.0.0 last I tried, but kohya gotten updates since then pip3 install --upgrade -r requirements.txt pip3 uninstall tensorflow && pip3 install tensorflow-rocm pip uninstall bitsandbytes && git clone https://github.com/broncotc/bitsandbytes-rocm # bitsandbytes not required if not using adam8? cd bitesandbytes-rocm && nano Makefile # Replace all 3 instances of 5.3.0 with 5.4.3 make hip python3 setup.py install

There are a lot of ROCm versions of bitsandbytes. For example this one: https://github.com/broncotc/bitsandbytes-rocm The problem is compatibility with most of the requirements. Kobold does a better job than ooba in offering a more streamlined approach for AMD users.