kohya-trainer VS EveryDream

This longer colab notebook: I did use this one (or one of the slight derivatives of it) and got out a safetensors file, but the lora didn't work at all--I'd use it a increase it's weight but I just would see no effect

It isn't what you are asking for (sry) but I struggled with this thing for way too long until I found out about the Kohya Trainer. https://github.com/Linaqruf/kohya-trainer So much easier with a lot of videos by the various YT folks. Standalone WebUI that just works. Life is good here!

It's seem that he is stable diffusion model creators. In that space, it's less knowing about the code and more experimenting on what would happen in the training. The stable diffusion community has repertoire of fine-tuning tools that is accessible for someone who have no single idea on the code behind it, no different than using application like kohya.

Have you tried out one of the colabs from https://github.com/Linaqruf/kohya-trainer ? The colabs themselves are pretty long, but you just have to read each step and then usually push the button to run that cell, then move on to the next one.

** https: //github.com/linaqruf/kohya-trainer**

There are a lot of variables that affect what kind of settings to use, but afaik the best solution to finding the right step count for what your training is still just to save multiple epochs and then run a x/y/z plot comparison. If you can't do that locally because of your 4gb card, you could try using Lora colabs that include inference capabilities.

You seem to be a little confused. You wont find an ipynb of a model. You would reference a model via a content portal like hugginface. If your model is hosted there, you dont have to download it to your computer or gdrive first. You just reference it with the hugginface-style reference, ie runwayml/stable-diffusion-v1-5. Some colabs will let you also reference a URL to pull down the model. Example. https://github.com/Linaqruf/kohya-trainer/blob/main/kohya-LoRA-dreambooth.ipynb. In that case, you can get the direct url to a checkpoint, for example at civit.ai. If you're decent at messing around with code, you can deconstruct that code block to use in a different colab. As for gdrive, it's only a couple dollars to get 100G.

Try this colab to train Lora, it can generate image without the UI too

Before I keep experimenting, any prior art here or tips? I considered splitting the high res squares up into 512x512s manually and training with them alongside the full picture 512x512s (maybe with "close up" added to caption?), but that's yet more work. Taking a look at https://github.com/victorchall/EveryDream to see if it might be a better fit.

In terms of training, you've got some really good links and comments to youtube tutorials, but if you're interested in more information about finetuning a model (as opposed to training from scratch), this is a good repo that has a lot of tools for finetuning, including an auto-captioner using BLIP and automatic file renaming. This is the actual finetuning repo.

Then I either use EveryDream's autocaptioner or the BLIP autocaptioner in the webui, use EveryDream's filename replacer to remove references to "a painting of", manually correct or touch up the captions that need it, then fire up the training tab in the webui.

Oh yeah, very good results. Using victorchall/EveryDream: Advanced fine tuning tools for vision models (github.com) I was able to train 12 subjects into ProtogenX5.3 with 25 images each (~300) + another ~300 "ground truth" images from LAION-5B and ffhq. Everydream requires you to caption every image, which is quite time consuming, but the results are pretty good. As with everything AI there's a lot of trial and error, though, especially when it comes to figuring out when the model's "done".

Probably something like this: https://github.com/victorchall/EveryDream

I use EveryDream. It is pretty technical to set up, but probably the only way to go if you're interested in large scale model training. It should work well for small scale stuff too, however I haven't done as much with that yet. I've mostly used one of the various dreambooth repos for single subject training.

There's a variety of methods that you could try, the basic one could be using textual inversion, you could find a lot of well explained tutorials on YouTube. Not so many people use it due to dreambooth and hypernetworks are more accessible but the option is there and you could use as a last resort. The next and probably the most popular one is dreambooth, again there's a lot of well explained tutorials on YouTube and depending on your situation (if you have a powerful GPU or you are willing to pay 0.33$/hr to rent a 3090) you could find alternatives like TheLastBen Colab Notebook version that allows you to train for free on Google Colab (with anime related stuff I personally didn't have to much success but I see other people having decent results, personally I get better results using Joe Penna but for you use case I think you may want to try other option) Next one are hypernetworks, easy to train using automatic1111, again a lot of tutorials on YouTube. From my experience it's a 50/50 I haven't tried that much so a have nothing more to say Last one and I think it might be more suitable for you but it requires more manual work than the other ones and I haven't seen any YouTube tutorial so you have to figure out things for yourself but is EveryDream. You can see it like Dreambooth+Textual inversion, like I said it requires a lot more work and a powerful GPU like Joe Penna repo but I think it's the best to train multiple stuff without making the model "forget" what it originally knows. You can see an example of how good it is in this other post