exllama

Open-source projects categorized as exllama
Edit details
Language: + Python + C++
Topics: llm llama llamacpp GUI ggml
Power Real-Time Data Analytics at Scale
Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
www.influxdata.com
featured

Top 4 exllama Open-Source Projects

exllama

  • booster

    Booster - open platform for serving LLM models (by gotzmann)

    • Project mention: Show HN: Collider – the platform for local LLM debug and inference at warp speed | news.ycombinator.com | 2023-11-30

  • Scout Monitoring

    Free Django app performance insights with Scout Monitoring. Get Scout setup in minutes, and let us sweat the small stuff. A couple lines in settings.py is all you need to start monitoring your apps. Sign up for our free tier today.

    Scout Monitoring logo

  • llama-api

    An OpenAI-like LLaMA inference API

    • Project mention: What do you use to run your models? | /r/LocalLLaMA | 2023-12-07

    https://github.com/c0sogi/llama-api , right? This offers better performance on GPU-optimized models, right?

  • magi_llm_gui

    A Qt GUI for large language models

    • Project mention: What is the best text web ui currently? | /r/LocalLLaMA | 2023-07-04

    Other than Ooba, this is my fav (and works with a TON of model architectures) -> https://github.com/shinomakoi/magi_llm_gui

  • sliced_llama

    Simple LLM inference server

    • Project mention: Mixture-of-Depths: Dynamically allocating compute in transformers | news.ycombinator.com | 2024-04-08

    There are already some implementations out there which attempt to accomplish this!

    Here's an example: https://github.com/silphendio/sliced_llama

    A gist pertaining to said example: https://gist.github.com/silphendio/535cd9c1821aa1290aa10d587...

    Here's a discussion about integrating this capability with ExLlama: https://github.com/turboderp/exllamav2/pull/275

    And same as above but for llama.cpp: https://github.com/ggerganov/llama.cpp/issues/4718#issuecomm...

NOTE: The open source projects on this list are ordered by number of github stars. The number of mentions indicates repo mentiontions in the last 12 Months or since we started tracking (Dec 2020).

exllama discussion

Log in or Post with

exllama related posts

  • What do you use to run your models?

    14 projects | /r/LocalLLaMA | 7 Dec 2023

Index

What are some of the best open-source exllama projects? This list will help you:

Project Stars
1 booster 129
2 llama-api 108
3 magi_llm_gui 40
4 sliced_llama 15

Sponsored
Free Django app performance insights with Scout Monitoring
Get Scout setup in minutes, and let us sweat the small stuff. A couple lines in settings.py is all you need to start monitoring your apps. Sign up for our free tier today.
www.scoutapm.com