Ask HN: Open-source GPT-3 alternatives

• nanoGPT

  69 32,050 5.4 Python

  The simplest, fastest repository for training/finetuning medium-sized GPTs.

What's the state of the art in GPT-3 alternatives right now, in practical terms? If your typical use case is taking a pretrained model and fine tuning it to a specific task, which LLM would yield the best results while running on consumer hardware? Note that I'm specifically asking for software that I can run on my own hardware, I'm not interested in paying OpenAI $0.02 per API request.

I'll start the recommendations with Karpathy's nanoGPT: https://github.com/karpathy/nanoGPT

What else do we have?

• RWKV-LM

  84 11,704 8.8 Python

  RWKV is an RNN with transformer-level LLM performance. It can be directly trained like a GPT (parallelizable). So it's combining the best of RNN and transformer - great performance, fast inference, saves VRAM, fast training, "infinite" ctx_len, and free sentence embedding.

• InfluxDB

  www.influxdata.com featured

  Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

  

• lm-human-preferences

  8 1,113 2.7 Python

  Code for the paper Fine-Tuning Language Models from Human Preferences

  

• ChatRWKV

  28 9,291 8.3 Python

  ChatRWKV is like ChatGPT but powered by RWKV (100% RNN) language model, and open source.

Suggest a related project