LoRA-EXTRACTOR
kohya_ss
LoRA-EXTRACTOR | kohya_ss | |
---|---|---|
8 | 132 | |
73 | 8,306 | |
- | - | |
10.0 | 9.9 | |
about 1 year ago | 7 days ago | |
Python | Python | |
- | Apache License 2.0 |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
LoRA-EXTRACTOR
-
The duality of this subreddit
Regardless of the method used to fine tune the model it wouldn't exceed the sizes I listed above. (fun fact, the maximum dimensions for LoRA is 320 (which limits the LoRA models to ~500mb, because that is the size of the smallest tensor) https://github.com/sashaok123/LoRA-EXTRACTOR/issues/8 (with LoCons it might be slightly larger, but I have not check out the theoretical limit of LoCons yet)
-
LORA Extraction from Custom Models in Google Colab
I changed sashaok123's code to work on Google Colab. https://github.com/AlirezaF80/LoRA-EXTRACTOR-Colab
-
Lora extractions VS. DB model - Can we truly replace 2/4gb models with 300mb Loras for the same results?
Can you extract a "LoRA transform" by comparing the weights from 2 checkpoints? Yes you can. https://github.com/sashaok123/LoRA-EXTRACTOR/blob/main/lib/extract_lora_from_models.py is a script that does exactly that, and if you look at the code, all it is is just applying singular value decomposition (SVD) (the underlying method behind principal component analysis (PCA), for those who are more Statistics/ML minded) to calculate the transformation between the two. Will you lose information when you "compress" the network using SVD? yes, the amount of "information" (in stats this is measured by variance) explained by the "lower ranked" component omitted by the new compressed representation, however the more similar the networks are, the less components will be needed to represent the differences.
-
Automatic1111 Webui Help: Dreambooth + DeepSpeed LoRA Training on 8GB VRAM
Instead of using kohya_ss, I used https://github.com/sashaok123/LoRA-EXTRACTOR
- LORA extractor tool
-
Can someone request AUTO1111 to include LORA-extraction script by sashaok123?
Here is the link to the LORA-extraction script on github by sashaok123: https://github.com/sashaok123/LoRA-EXTRACTOR
-
Created a tiny script for more convenient extraction of LoRA models
Link to GitHub, where you can download the script.
kohya_ss
-
Some semi-advanced LoRA & kohya_ss questions
Many of the options are explained here https://github.com/bmaltais/kohya_ss/wiki/LoRA-training-parameters
-
Lora training with Kohya issue
training in BF16 might solve this issue from what I saw in this ticket. I know other people ran into the issue too https://github.com/bmaltais/kohya_ss/issues/1382
-
What is the best way to merge multiple loras in to one model?
for lycoris loras you can use the command-line script from the kohya-ss repo https://github.com/bmaltais/kohya_ss/blob/master/networks/merge_lora.py i have an older version checked out from late july, it had a separate merge_lycoris.py for for this purpose, it's probably unified now in a single file
- Evidence that LoRA extraction in Kohya is broken?
-
Merging Lora with Checkpoint Model?
I usually do that with kohya_ss, a tool made for making LoRAs and finetunes. It might be a bit of a pain to set up just to do this one task, but if nobody gives you an easier method, look into it. https://github.com/bmaltais/kohya_ss
-
How I got Kohya_SS working on Arch Linux, including an up-to-date pip requirements file
After that, make your staging directory, and do the git clone https://github.com/bmaltais/kohya_ss.git, and navigate inside of it. Now, here's where things can become a pain. I used pyenv to set my system level python to 3.10.6 with pyenv global 3.10.6, though you can probably just use "local" and do it for the current shell. You NEED it to be active however before you set up your venv. If you do python --version and get 3.10.6, you're ready for this next part. Make your venv with python -m venv venv. This is the simplest way, it'll create a virtual environment in your current folder named venv. You'll do a source venv/bin/activate and then do which python to make sure it's using the python from the venv. Now for the fun part. The included setup scripts have been flaky for me, so I just went through the requirements and installed everything by hand. I'm going to do this guide right now for nvidia, because I just got a 4090 for this stuff. If this ends up working well for others and there's demand, I'll try to reproduce this for AMD (But I'll be honest, I got an nvidia card because bitsandbytes doesn't have full rocm support, nor do most libraries, so it's not very reliable). After installing everything and testing it works at least at a basic level for dreambooth training, my finished requirements.txt for pip is as below:
-
The best open source LoRA model training tools
Earlier I created a post where I asked for recommendations for LoRA model training tutorials. The first one I looked at used the kohya_ss GUI. That GitHub repo already has two tutorials, which are quite good, so I ended up using those:
-
Script does...nothing
I have tried my best to research this issue and have not come up with much. It is obvious that its a backend issue right? The guides that I used https://github.com/bmaltais/kohya_ss and https://github.com/pyenv-win/pyenv-win/
- Using LoRa on SDXL 1.0 (not using the Kohra GUIs)
-
How do I reduce the size of my Lora models?
I am training on a 12GB 3060 using kohya_ss. Is there a setting or something I'm missing?
What are some alternatives?
LoRA-EXTRACTOR-Colab - A small script to extract LoRA models from custom checkpoints, in Google Colab.
sd_dreambooth_extension
sd-scripts
EveryDream-trainer - General fine tuning for Stable Diffusion
stable-diffusion-webui - Stable Diffusion web UI
automatic - SD.Next: Advanced Implementation of Stable Diffusion and other Diffusion-based generative image models
kohya_ss_colab - a (successful) attepmt to port kohya_ss to colab
LoRA_Easy_Training_Scripts - A UI made in Pyside6 to make training LoRA/LoCon and other LoRA type models in sd-scripts easy
sd-webui-additional-networks
LyCORIS - Lora beYond Conventional methods, Other Rank adaptation Implementations for Stable diffusion.
kohya-sd-scripts-webui - Gradio wrapper for sd-scripts by kohya