You can now train a 70B language model at home

• litgpt

  1 6,625 9.8 Python

  Pretrain, finetune, deploy 20+ LLMs on your own data. Uses state-of-the-art techniques: flash attention, FSDP, 4-bit, LoRA, and more.

  

Lit-GPT is what I have been using to pretrain models at home: https://github.com/Lightning-AI/litgpt

• lora

  83 6,597 0.0 Jupyter Notebook

  Using Low-rank adaptation to quickly fine-tune diffusion models. (by cloneofsimo)

Diffusion unet has an "extended" version nowadays that applies to the resnet part as well as the cross-attention: https://github.com/cloneofsimo/lora

• InfluxDB

  www.influxdata.com sponsored

  Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

  

• hivemind

  40 1,837 5.4 Python

  Decentralized deep learning in PyTorch. Built to train models on thousands of volunteers across the world.

  

https://github.com/learning-at-home/hivemind is also relevant

Suggest a related project