How does using QLoRAs when running Llama on CPU work?

• peft

  26 13,877 9.7 Python

  🤗 PEFT: State-of-the-art Parameter-Efficient Fine-Tuning.

  

It seems like the merge_and_unload function in this PEFT script might be what they are referring to: https://github.com/huggingface/peft/blob/main/src/peft/tuners/lora.py

• FastChat

  83 34,277 9.6 Python

  An open platform for training, serving, and evaluating large language models. Release repo for Vicuna and Chatbot Arena.

  

There is a script to merge lora or qlora weights with llama origin weights. https://github.com/lm-sys/FastChat/blob/main/fastchat/model/apply_lora.py

• InfluxDB

  www.influxdata.com featured

  Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

  

Suggest a related project