[P] minLoRA: An Easy-to-Use PyTorch Library for Applying LoRA to PyTorch Models

• minLoRA

  3 388 2.4 Jupyter Notebook

  minLoRA: a minimal PyTorch library that allows you to apply LoRA to any PyTorch model.

Theirs requires you to rewrite the whole model and replace every layer you want to apply LoRA to to the LoRA counterpart, or use monky-patching. Mine utilizes PyTorch parametrizations to inject the LoRA logic to existing models. If your model has nn.Linear, you can call add_lora(model) to add LoRA to all the linear layers. And it's not limited to Linear, you can see how I extended it to Embedding, Conv2d in a couple lines of code. https://github.com/cccntu/minLoRA/blob/main/minlora/model.py

• peft

  26 13,783 9.7 Python

  🤗 PEFT: State-of-the-art Parameter-Efficient Fine-Tuning.

  

Sorry, but why do we need another package? Can't you build on top of https://github.com/huggingface/peft ?

• InfluxDB

  www.influxdata.com featured

  Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

  

Suggest a related project