LoRA-EXTRACTOR
A small script to facilitate the extraction of LoRA models from custom checkpoints. (by sashaok123)
sd-scripts
By kohya-ss
LoRA-EXTRACTOR | sd-scripts | |
---|---|---|
8 | 64 | |
73 | 4,199 | |
- | - | |
10.0 | 9.7 | |
about 1 year ago | 5 days ago | |
Python | Python | |
- | Apache License 2.0 |
The number of mentions indicates the total number of mentions that we've tracked plus the number of user suggested alternatives.
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
LoRA-EXTRACTOR
Posts with mentions or reviews of LoRA-EXTRACTOR.
We have used some of these posts to build our list of alternatives
and similar projects. The last one was on 2023-03-03.
-
The duality of this subreddit
Regardless of the method used to fine tune the model it wouldn't exceed the sizes I listed above. (fun fact, the maximum dimensions for LoRA is 320 (which limits the LoRA models to ~500mb, because that is the size of the smallest tensor) https://github.com/sashaok123/LoRA-EXTRACTOR/issues/8 (with LoCons it might be slightly larger, but I have not check out the theoretical limit of LoCons yet)
-
LORA Extraction from Custom Models in Google Colab
I changed sashaok123's code to work on Google Colab. https://github.com/AlirezaF80/LoRA-EXTRACTOR-Colab
-
Lora extractions VS. DB model - Can we truly replace 2/4gb models with 300mb Loras for the same results?
Can you extract a "LoRA transform" by comparing the weights from 2 checkpoints? Yes you can. https://github.com/sashaok123/LoRA-EXTRACTOR/blob/main/lib/extract_lora_from_models.py is a script that does exactly that, and if you look at the code, all it is is just applying singular value decomposition (SVD) (the underlying method behind principal component analysis (PCA), for those who are more Statistics/ML minded) to calculate the transformation between the two. Will you lose information when you "compress" the network using SVD? yes, the amount of "information" (in stats this is measured by variance) explained by the "lower ranked" component omitted by the new compressed representation, however the more similar the networks are, the less components will be needed to represent the differences.
-
Automatic1111 Webui Help: Dreambooth + DeepSpeed LoRA Training on 8GB VRAM
Instead of using kohya_ss, I used https://github.com/sashaok123/LoRA-EXTRACTOR
- LORA extractor tool
-
Can someone request AUTO1111 to include LORA-extraction script by sashaok123?
Here is the link to the LORA-extraction script on github by sashaok123: https://github.com/sashaok123/LoRA-EXTRACTOR
-
Created a tiny script for more convenient extraction of LoRA models
Link to GitHub, where you can download the script.
sd-scripts
Posts with mentions or reviews of sd-scripts.
We have used some of these posts to build our list of alternatives
and similar projects. The last one was on 2023-11-16.
- Everything you know about loss is a lie
- Evidence that LoRA extraction in Kohya is broken?
- Stable Diffusion XL (SDXL) DreamBooth training with EMA (Exponential Moving Average) on the way
-
Installing kohya_ss GUI on AWS
This repository mostly provides a Windows-focused Gradio GUI for Kohya's Stable Diffusion trainers... but support for Linux OS is also provided through community contributions.
- Question on SD Finetuning
-
Trying to put up a simple dreambooth for sdxl, but an errors pops up
Leaving this here because i'm very tired, so this is the file of the ipynb that uses the sdxl_train.py from the https://github.com/kohya-ss/sd-scripts/tree/sdxl repo, if anybody find out why when getting to the training i get this very empty error : " [00:09:11] WARNING The following values were not passed to "
-
Finally SDXL coming to the Automatic1111 Web UI
You can try and test training LoRAs now https://github.com/kohya-ss/sd-scripts/tree/sdxl
-
Help with LORA Training - Kohya_ss Regularization
This might help.
-
need a lora traning guide for linux
Kohya_ss sd-scripts Seems to be the standard for lora training. The linked page has an English translation, but doesn't really have system specific tips. Someone else has a popular gui for it, but it's designed with windows in mind. There's another, simpler gui, but its still in development and the dev doesn't do any testing on Linux. With any of these, I run into dependency conflicts like crazy.
-
SDXL 0.9 is wild but trying to imagine where we go from here is breaking my brain.
"Direct training" is already feasible with masking in kohya-ss: https://github.com/kohya-ss/sd-scripts/pull/589
What are some alternatives?
When comparing LoRA-EXTRACTOR and sd-scripts you can also consider the following projects:
LoRA-EXTRACTOR-Colab - A small script to extract LoRA models from custom checkpoints, in Google Colab.
kohya_ss
sd_dreambooth_extension
stable-diffusion-webui - Stable Diffusion web UI
ComfyUI - The most powerful and modular stable diffusion GUI, api and backend with a graph/nodes interface.
bitsandbytes-rocm
lora - Using Low-rank adaptation to quickly fine-tune diffusion models.
kohya-trainer - Adapted from https://note.com/kohya_ss/n/nbf7ce8d80f29 for easier cloning
EveryDream2trainer
LoRA_Easy_Training_Scripts - A UI made in Pyside6 to make training LoRA/LoCon and other LoRA type models in sd-scripts easy
LoRA - Code for loralib, an implementation of "LoRA: Low-Rank Adaptation of Large Language Models"
LoRA-EXTRACTOR vs LoRA-EXTRACTOR-Colab
sd-scripts vs kohya_ss
LoRA-EXTRACTOR vs kohya_ss
sd-scripts vs sd_dreambooth_extension
LoRA-EXTRACTOR vs stable-diffusion-webui
sd-scripts vs ComfyUI
sd-scripts vs bitsandbytes-rocm
sd-scripts vs lora
sd-scripts vs kohya-trainer
sd-scripts vs EveryDream2trainer
sd-scripts vs LoRA_Easy_Training_Scripts
sd-scripts vs LoRA