Mlc-llm Alternatives

Similar projects and alternatives to mlc-llm

• text-generation-webui

  876 35,862 9.9 Python mlc-llm VS text-generation-webui

  A Gradio web UI for Large Language Models. Supports transformers, GPTQ, AWQ, EXL2, llama.cpp (GGUF), Llama models.

• llama.cpp

  769 55,846 10.0 C++ mlc-llm VS llama.cpp

  LLM inference in C/C++

• WorkOS

  workos.com sponsored

  The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.

  

• ROCm

  198 3,637 0.0 Python mlc-llm VS ROCm

  Discontinued AMD ROCm™ Software - GitHub Home [Moved to: https://github.com/ROCm/ROCm]

  

• ollama

  192 58,943 9.9 Go mlc-llm VS ollama

  Get up and running with Llama 3, Mistral, Gemma, and other large language models.

  

• whisper.cpp

  187 30,942 9.8 C mlc-llm VS whisper.cpp

  Port of OpenAI's Whisper model in C/C++

• koboldcpp

  180 3,749 10.0 C++ mlc-llm VS koboldcpp

  A simple one-file way to run various GGML and GGUF models with KoboldAI's UI

• FastChat

  82 33,877 9.6 Python mlc-llm VS FastChat

  An open platform for training, serving, and evaluating large language models. Release repo for Vicuna and Chatbot Arena.

  

• InfluxDB

  www.influxdata.com sponsored

  Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

  

• FLiPStackWeekly

  79 14 9.9 mlc-llm VS FLiPStackWeekly

  FLaNK AI Weekly covering Apache NiFi, Apache Flink, Apache Kafka, Apache Spark, Apache Iceberg, Apache Ozone, Apache Pulsar, and more...

• ggml

  69 9,642 9.8 C mlc-llm VS ggml

  Tensor library for machine learning

• GPTQ-for-LLaMa

  75 2,913 8.6 Python mlc-llm VS GPTQ-for-LLaMa

  4 bits quantization of LLaMA using GPTQ

• exllama

  64 2,582 9.0 Python mlc-llm VS exllama

  A more memory-efficient rewrite of the HF transformers implementation of Llama for use with quantized weights.

• llama-cpp-python

  54 6,378 9.9 Python mlc-llm VS llama-cpp-python

  Python bindings for llama.cpp

• dalai

  59 13,044 6.5 CSS mlc-llm VS dalai

  The simplest way to run LLaMA on your local machine

• open_llama

  52 7,193 5.3 mlc-llm VS open_llama

  OpenLLaMA, a permissively licensed open source reproduction of Meta AI’s LLaMA 7B trained on the RedPajama dataset

  

• web-llm

  42 9,018 9.0 TypeScript mlc-llm VS web-llm

  Bringing large-language models and chat to web browsers. Everything runs inside the browser with no server support.

  

• jsonformer

  25 3,774 5.4 Jupyter Notebook mlc-llm VS jsonformer

  A Bulletproof Way to Generate Structured JSON from Language Models

• sparsegpt

  16 620 3.2 Python mlc-llm VS sparsegpt

  Code for the ICML 2023 paper "SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shot".

  

• CTranslate2

  13 2,776 9.0 C++ mlc-llm VS CTranslate2

  Fast inference engine for Transformer models

  

• tvm

  15 11,156 9.9 Python mlc-llm VS tvm

  Open deep learning compiler stack for cpu, gpu and specialized accelerators

  

• Open-Llama

  7 637 10.0 Python mlc-llm VS Open-Llama

  Discontinued The complete training code of the open-source high-performance Llama model, including the full process from pre-training to RLHF.

• SaaSHub

  www.saashub.com sponsored

  SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives