New Open-source LLMs! 🤯 The Falcon has landed! 7B and 40B

• sparsegpt

  16 624 2.4 Python

  Code for the ICML 2023 paper "SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shot".

  

There is this : https://github.com/IST-DASLab/sparsegpt

• trl

  13 8,120 9.7 Python

  Train transformer language models with reinforcement learning.

  

For lora - PEFT seems to work. I don't have patience to wait 5 hours, but modifying this example seems to work. You don't even need to modify that much, as their model just as neo-x uses query_key_value name for self-attention.

• InfluxDB

  www.influxdata.com featured

  Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

  

Suggest a related project