Text-generation-inference Alternatives

Similar projects and alternatives to text-generation-inference

• text-generation-webui

  876 35,583 9.9 Python text-generation-inference VS text-generation-webui

  A Gradio web UI for Large Language Models. Supports transformers, GPTQ, AWQ, EXL2, llama.cpp (GGUF), Llama models.

• llama.cpp

  766 55,117 9.9 C++ text-generation-inference VS llama.cpp

  LLM inference in C/C++

• WorkOS

  workos.com sponsored

  The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.

  

• stable-diffusion

  382 65,243 0.0 Jupyter Notebook text-generation-inference VS stable-diffusion

  A latent text-to-image diffusion model

  

• Open-Assistant

  329 36,601 9.1 Python text-generation-inference VS Open-Assistant

  OpenAssistant is a chat-based assistant that understands tasks, can interact with third-party systems, and retrieve information dynamically to do so.

  

• ollama

  187 55,134 9.9 Go text-generation-inference VS ollama

  Get up and running with Llama 2, Mistral, Gemma, and other large language models.

  

• llama

  184 52,244 8.2 Python text-generation-inference VS llama

  Inference code for Llama models

  

• LocalAI

  82 19,033 9.9 C++ text-generation-inference VS LocalAI

  :robot: The free, Open Source OpenAI alternative. Self-hosted, community-driven and local-first. Drop-in replacement for OpenAI running on consumer-grade hardware. No GPU required. Runs gguf, transformers, diffusers and many more models architectures. It allows to generate Text, Audio, Video, Images. Also with voice cloning capabilities.

  

• InfluxDB

  www.influxdata.com sponsored

  Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

  

• FLiPStackWeekly

  78 12 9.9 text-generation-inference VS FLiPStackWeekly

  FLaNK AI Weekly covering Apache NiFi, Apache Flink, Apache Kafka, Apache Spark, Apache Iceberg, Apache Ozone, Apache Pulsar, and more...

• exllama

  64 2,566 9.0 Python text-generation-inference VS exllama

  A more memory-efficient rewrite of the HF transformers implementation of Llama for use with quantized weights.

• ClickBench

  68 567 9.1 HTML text-generation-inference VS ClickBench

  ClickBench: a Benchmark For Analytical Databases

  

• llama-cpp-python

  54 6,280 9.9 Python text-generation-inference VS llama-cpp-python

  Python bindings for llama.cpp

• bitsandbytes

  61 5,344 9.4 Python text-generation-inference VS bitsandbytes

  Accessible large language models via k-bit quantization for PyTorch.

• DeepSpeed

  51 32,447 9.8 Python text-generation-inference VS DeepSpeed

  DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.

  

• safetensors

  31 2,408 8.4 Python text-generation-inference VS safetensors

  Simple, safe way to store and distribute tensors

  

• vllm

  30 17,656 9.9 Python text-generation-inference VS vllm

  A high-throughput and memory-efficient inference and serving engine for LLMs

  

• refact

  33 1,404 9.7 JavaScript text-generation-inference VS refact

  WebUI for Fine-Tuning and Self-hosting of Open-Source Large Language Models for Coding

  

• server

  24 7,277 9.5 Python text-generation-inference VS server

  The Triton Inference Server provides an optimized cloud and edge inferencing solution. (by triton-inference-server)

  

• metaflow

  24 7,559 9.2 Python text-generation-inference VS metaflow

  :rocket: Build and manage real-life ML, AI, and data science projects with ease!

  

• openvino

  17 5,864 10.0 C++ text-generation-inference VS openvino

  OpenVINO™ is an open-source toolkit for optimizing and deploying AI inference

  

• basaran

  22 1,281 10.0 Python text-generation-inference VS basaran

  Discontinued Basaran is an open-source alternative to the OpenAI text completion API. It provides a compatible streaming API for your Hugging Face Transformers-based text generation models.

  

• SaaSHub

  www.saashub.com sponsored

  SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives