lora
Our great sponsors
sd_dreambooth_extension | lora | |
---|---|---|
115 | 83 | |
1,818 | 6,597 | |
- | - | |
9.0 | 0.0 | |
about 1 month ago | about 1 month ago | |
Python | Jupyter Notebook | |
GNU General Public License v3.0 or later | Apache License 2.0 |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
sd_dreambooth_extension
- SDXL Training for Auto1111 is now Working on a 24GB Card
-
(Requesting Help)
I am trying to use StableDiffusion via AUTOMATIC1111 with the Dreambooth extension
-
it will be an absolute madness when sdxl becomes standard model and we start getting other models from it
When I first attempted SD training, I was very frustrated. It wasn't until I found this obscure forum thread on Github that I actually started producing great results with Dreambooth. Because I have such satisfactory results, I'm very reluctant to beat my brains against LoRa and its related training techniques. I gave up trying to train TI embeddings a long time ago. And I never figured out how to train or how to use hypernetworks. I've only been able to get good results with Dreambooth directly because of that thread I linked above. I make LoRas by extracting them from Dreambooth-trained checkpoints. And I have no idea if I'm doing the extractions the right way or not.
-
"Exception training model: ' Some tensors share memory" with Dreambooth on Vladmatic
Getting the same with automatic1111 and sd_dreambooth extension. Check out more here in the issues log: https://github.com/d8ahazard/sd_dreambooth_extension/issues/1266
-
Yo, DreamBooth gatekeepers, SHARE YOUR HYPERPARAMETERS, please.
It's several moths old and many things have changed. But the spreadsheet available through this thread on Github has been indispensable for me when I train Dreambooth models. I'm astounded no one talks about it. I bring it up all the time. The research presented there should be continued. I'd love to see similar research done for SD v2.1.
-
What is the BEST solution for hyper realistic person training?
Training rate is paramount. Read this Github thread.
-
How do you train your LoRAs, 1 Epoch or >1 Epoch (same # of steps)?
https://github.com/d8ahazard/sd_dreambooth_extension/discussions/547/ (in depth training principles understanding)
-
Struggling to install Dreambooth
sd_dreambooth_extension https://github.com/d8ahazard/sd_dreambooth_extension.git main 926ae204 Fri Mar 31 15:12:45 2023 unknown
- Attempting to train a lora with RTX 2060 6 GB vRAM, how to go about this?
-
SD just released an open source version of their GUI called StableStudio
also the Dreambooth extension supports API (https://github.com/d8ahazard/sd_dreambooth_extension/blob/main/scripts/api.py) so i'm not sure where do you get those news :/
lora
-
You can now train a 70B language model at home
Diffusion unet has an "extended" version nowadays that applies to the resnet part as well as the cross-attention: https://github.com/cloneofsimo/lora
-
How it feels right now
Absolutely. But that doesn't matter because you only have to train it at scale, once. There are papers released already that show it's possible to update weights in small sections. You won't have to wait for the next monolithic LLM to drop to get up to date information. It will start to learn in bits and pieces.
-
LoRA tuning in julia
No, it's a deep learning thing
-
What does Lora mean?
Low Rank Adaptation of Large Language Models.
-
[D] An ELI5 explanation for LoRA - Low-Rank Adaptation.
Recently, I have seen the LoRA technique (Low-Rank Adaptation of Large Language Models) as a popular method for fine-tuning LLMs and other models.
-
Combining LoRA, Retro, and Large Language Models for Efficient Knowledge Retrieval and Retention
Enter LoRA, a method proposed for adapting pre-trained models to specific tasks[2]. By freezing pre-trained model weights and injecting trainable rank decomposition matrices into the transformer architecture, LoRA can reduce the number of trainable parameters and the GPU memory requirement, making the adaptation of LLMs for downstream tasks more feasible.
-
100K Context Windows
Open-source LLM projects have largely solved this using Low-Rank Adaptation of Large Language Models (LoRA): https://arxiv.org/abs/2106.09685
Apparently an RTX 4090 running overnight is sufficient to produce a fine-tuned model that can spit out new Harry Potter stories, or whatever...
-
President Biden meets with AI CEOs at the White House amid ethical criticism
Alpaca was trained for $600 ($100 for the smaller model) and offers outputs competitive with ChatGTP. https://arxiv.org/abs/2106.09685
- LoRA: Low-Rank Adaptation of Large Language Models
- LORA: Low-Rank Adaptation of Large Language Models
What are some alternatives?
kohya_ss
stable-diffusion-webui - Stable Diffusion web UI
kohya-trainer - Adapted from https://note.com/kohya_ss/n/nbf7ce8d80f29 for easier cloning
LyCORIS - Lora beYond Conventional methods, Other Rank adaptation Implementations for Stable diffusion.
stable-diffusion-webui-wd14-tagger - Labeling extension for Automatic1111's Web UI
dreambooth-training-guide
ControlNet - Let us control diffusion models!
sd-scripts
sd-webui-additional-networks
sd-webui-controlnet - WebUI extension for ControlNet
diffusers - 🤗 Diffusers: State-of-the-art diffusion models for image and audio generation in PyTorch and FLAX.