Tiny models for contextually coherent conversations?

• FastChat

  83 34,514 9.6 Python

  An open platform for training, serving, and evaluating large language models. Release repo for Vicuna and Chatbot Arena.

  

Don't think so, there's an issue created around this https://github.com/lm-sys/FastChat/issues/925

• RWKV-Runner

  4 4,595 9.7 TypeScript

  A RWKV management and startup tool, full automation, only 8MB. And provides an interface compatible with the OpenAI API. RWKV is a large language model that is fully open source and available for commercial use.

• InfluxDB

  www.influxdata.com featured

  Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

  

• rwkv.cpp

  12 1,111 6.8 C++

  INT4/INT5/INT8 and FP16 inference on CPU for RWKV language model

  

• mlc-llm

  89 17,150 9.9 Python

  Enable everyone to develop, optimize and deploy AI models natively on everyone's devices.

  

MLC-LLM Vicuna is a 7B model that fits into about 2gb of Vram and can make small talk with context. MLC LLM | Home

• koboldcpp

  180 3,951 10.0 C++

  A simple one-file way to run various GGML and GGUF models with KoboldAI's UI

As a bit of a fun fact, KoboldCpp also has support for non-LLaMA GGML models, which includes the RWKV conversion they just linked

• SaaSHub

  www.saashub.com featured

  SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives

  

Suggest a related project