Explore large language models on any computer with 512MB of RAM

• languagemodels

  9 1,161 9.4 Python

  Explore large language models in 512MB of RAM

• LaMini-LM

  9 801 7.8

  LaMini-LM: A Diverse Herd of Distilled Models from Large-Scale Instructions

  

• InfluxDB

  www.influxdata.com featured

  Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

  

• CTranslate2

  14 2,825 8.9 C++

  Fast inference engine for Transformer models

  

FLAN-T5 models generally perform well for their size, but they are encode-decoder models, and they aren't as widely supported for efficient inference. I wanted students to be able to run everything locally on CPU, so I was ideally hoping for something that supported quantization for CPU inference. I explored llama.cpp and GGML, but ultimately landed on ctranslate2 for inference.

Suggest a related project