mpt-30B-inference Alternatives

Similar projects and alternatives to mpt-30B-inference

• exllama

  64 2,624 9.0 Python mpt-30B-inference VS exllama

  A more memory-efficient rewrite of the HF transformers implementation of Llama for use with quantized weights.

• vllm

  31 19,672 9.9 Python mpt-30B-inference VS vllm

  A high-throughput and memory-efficient inference and serving engine for LLMs

  

• InfluxDB

  www.influxdata.com featured

  Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

  

• text-generation-inference

  29 8,053 9.6 Python mpt-30B-inference VS text-generation-inference

  Large Language Model Text Generation Inference

  

• rwkv.cpp

  12 1,113 6.8 C++ mpt-30B-inference VS rwkv.cpp

  INT4/INT5/INT8 and FP16 inference on CPU for RWKV language model

  

• inference

  2 2,799 9.8 Python mpt-30B-inference VS inference

  Replace OpenAI GPT with another LLM in your app by changing a single line of code. Xinference gives you the freedom to use any LLM you need. With Xinference, you're empowered to run inference with any open-source language models, speech recognition models, and multimodal models, whether in the cloud, on-premises, or even on your laptop.

  

• llm-rp

  2 24 10.0 Python mpt-30B-inference VS llm-rp

  Discontinued ✨ Your Custom Offline Role Play with LLM and Stable Diffusion on Mac and Linux (for now) 🧙‍♂️

• chatdocs

  11 650 6.9 Python mpt-30B-inference VS chatdocs

  Chat with your documents offline using AI.

• SaaSHub

  www.saashub.com featured

  SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives