text-generation-inference

Large Language Model Text Generation Inference (by huggingface)

Text-generation-inference Alternatives

Similar projects and alternatives to text-generation-inference

huggingface
text-generation-inference
  1. text-generation-webui

    A Gradio web UI for Large Language Models with support for multiple inference backends.

  2. Nutrient

    Nutrient - The #1 PDF SDK Library. Bad PDFs = bad UX. Slow load times, broken annotations, clunky UX frustrates users. Nutrient’s PDF SDKs gives seamless document experiences, fast rendering, annotations, real-time collaboration, 100+ features. Used by 10K+ devs, serving ~half a billion users worldwide. Explore the SDK for free.

    Nutrient logo
  3. llama.cpp

    LLM inference in C/C++

  4. ollama

    Get up and running with Llama 3.3, DeepSeek-R1, Phi-4, Gemma 2, and other large language models.

  5. Open-Assistant

    OpenAssistant is a chat-based assistant that understands tasks, can interact with third-party systems, and retrieve information dynamically to do so.

  6. llama

    Inference code for Llama models

  7. lapce

    Lightning-fast and Powerful Code Editor written in Rust

  8. Graal

    GraalVM compiles Java applications into native executables that start instantly, scale fast, and use fewer compute resources 🚀

  9. CodeRabbit

    CodeRabbit: AI Code Reviews for Developers. Revolutionize your code reviews with AI. CodeRabbit offers PR summaries, code walkthroughs, 1-click suggestions, and AST-based analysis. Boost productivity and code quality across all major languages with each PR.

    CodeRabbit logo
  10. LocalAI

    :robot: The free, Open Source alternative to OpenAI, Claude and others. Self-hosted and local-first. Drop-in replacement for OpenAI, running on consumer-grade hardware. No GPU required. Runs gguf, transformers, diffusers and many more models architectures. Features: Generate Text, Audio, Video, Images, Voice Cloning, Distributed, P2P inference

  11. FLiPStackWeekly

    FLaNK AI Weekly covering Apache NiFi, Apache Flink, Apache Kafka, Apache Spark, Apache Iceberg, Apache Ozone, Apache Pulsar, and more...

  12. Mage

    🧙 The modern replacement for Airflow. Mage is an open-source data pipeline tool for transforming and integrating data. https://github.com/mage-ai/mage-ai

  13. exllama

    A more memory-efficient rewrite of the HF transformers implementation of Llama for use with quantized weights.

  14. llama-cpp-python

    Python bindings for llama.cpp

  15. onnxruntime

    ONNX Runtime: cross-platform, high performance ML inferencing and training accelerator

  16. llm

    Access large language models from the command-line (by simonw)

  17. vllm

    A high-throughput and memory-efficient inference and serving engine for LLMs

  18. refact

    WebUI for Fine-Tuning and Self-hosting of Open-Source Large Language Models for Coding

  19. server

    The Triton Inference Server provides an optimized cloud and edge inferencing solution. (by triton-inference-server)

  20. openvino

    OpenVINO™ is an open source toolkit for optimizing and deploying AI inference

  21. optimum

    🚀 Accelerate inference and training of 🤗 Transformers, Diffusers, TIMM and Sentence Transformers with easy to use hardware optimization tools

  22. blog

    5 text-generation-inference VS blog

    Public repo for HF blog posts (by huggingface)

  23. SaaSHub

    SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives

    SaaSHub logo
NOTE: The number of mentions on this list indicates mentions on common posts plus user suggested alternatives. Hence, a higher number means a better text-generation-inference alternative or higher similarity.

text-generation-inference discussion

Log in or Post with

text-generation-inference reviews and mentions

Posts with mentions or reviews of text-generation-inference. We have used some of these posts to build our list of alternatives and similar projects. The last one was on 2024-06-05.

Stats

Basic text-generation-inference repo stats
30
9,756
9.8
4 days ago

Sponsored
Nutrient - The #1 PDF SDK Library
Bad PDFs = bad UX. Slow load times, broken annotations, clunky UX frustrates users. Nutrient’s PDF SDKs gives seamless document experiences, fast rendering, annotations, real-time collaboration, 100+ features. Used by 10K+ devs, serving ~half a billion users worldwide. Explore the SDK for free.
nutrient.io

Did you know that Python is
the 2nd most popular programming language
based on number of references?