You can now train a 70B language model at home

• litgpt

  1 6,944 9.8 Python

  Pretrain, finetune, deploy 20+ LLMs on your own data. Uses state-of-the-art techniques: flash attention, FSDP, 4-bit, LoRA, and more.

  

Lit-GPT is what I have been using to pretrain models at home: https://github.com/Lightning-AI/litgpt

• lora

  83 6,690 0.0 Jupyter Notebook

  Using Low-rank adaptation to quickly fine-tune diffusion models. (by cloneofsimo)

Diffusion unet has an "extended" version nowadays that applies to the resnet part as well as the cross-attention: https://github.com/cloneofsimo/lora

• Scout Monitoring

  www.scoutapm.com featured

  Free Django app performance insights with Scout Monitoring. Get Scout setup in minutes, and let us sweat the small stuff. A couple lines in settings.py is all you need to start monitoring your apps. Sign up for our free tier today.

  

• hivemind

  40 1,847 4.8 Python

  Decentralized deep learning in PyTorch. Built to train models on thousands of volunteers across the world.

  

https://github.com/learning-at-home/hivemind is also relevant

Suggest a related project