llmash

Ollama-compatible LLM server on a tuned llama.cpp fork. Same commands, API and model store as Ollama, up to 4x Ollama's decode speed and 2.8x vLLM's in our benchmarks, from custom CUDA kernels, MTP speculation and its own 3-bit builds. Uncensors models in one command. Windows and Linux, NVIDIA. (by omgitsbase)
Review Interface LLM ollama ollama-api vllm
Source Code
llmash Reviews
Suggest alternative
Edit details
SaaSHub - Software Alternatives and Reviews
SaaSHub helps you find the best software and product alternatives
www.saashub.com
sponsored

Llmash Alternatives

Similar projects and alternatives to llmash

omgitsbase
llmash

  1. tiny-vllm

    2 llmash VS tiny-vllm

    Build your own high performance LLM inference engine in C++ and CUDA - a smaller version of vLLM

  2. AppSignal

    Monitoring that respects your time & budget. APM, error tracking, and dashboards for modern web apps. Ten-minute setup, transparent flat pricing, and support from engineers who actually use the product.

    AppSignal logo

  3. voidllm

    3 llmash VS voidllm

    Privacy-first LLM proxy and AI gateway - load balancing, multi-provider routing, API key management, usage tracking, rate limiting. Self-hosted. Zero knowledge of your prompts.

  4. runanywhere-sdks

    4 llmash VS runanywhere-sdks

    Production ready toolkit to run AI locally

  5. WhiteLotus

    1 llmash VS WhiteLotus

    LLM inference engine built from scratch in C++. No PyTorch, no frameworks.

  6. Mooncake

    4 llmash VS Mooncake

    Mooncake is the serving platform for Kimi, a leading LLM service provided by Moonshot AI.

  7. vllm.cpp

    1 llmash VS vllm.cpp

    a community oriented 1:1, vLLM-alike (Continuous batching, paged KV) engine in C++ with additional features (GGUF, RadixAttention, Cache-aware scheduling, ...)

  8. booster

    1 llmash VS booster

    Booster - open accelerator for LLM models. Better inference and debugging for AI hackers (by gotzmann)

  9. SaaSHub

    SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives

    SaaSHub logo
NOTE: The number of mentions on this list indicates mentions on common posts plus user suggested alternatives. Hence, a higher number means a better llmash alternative or higher similarity.
Suggest an alternative to llmash

llmash discussion

Log in or Post with

llmash reviews and mentions

Posts with mentions or reviews of llmash. We have used some of these posts to build our list of alternatives and similar projects.

Stats

Basic llmash repo stats
4
15
9.5
8 days ago

omgitsbase/llmash is an open source project licensed under MIT License which is an OSI approved license.

The primary programming language of llmash is C++.

Popular Comparisons

  1. llmash VS tiny-vllm
  2. llmash VS voidllm
  3. llmash VS runanywhere-sdks
  4. llmash VS WhiteLotus
  5. llmash VS Mooncake
  6. llmash VS vllm.cpp
  7. llmash VS booster

Sponsored
Monitoring that respects your time & budget
APM, error tracking, and dashboards for modern web apps. Ten-minute setup, transparent flat pricing, and support from engineers who actually use the product.
www.appsignal.com

Did you know that C++ is
the 7th most popular programming language
based on number of references?

Loading...