Sliced_llama Alternatives

Similar projects and alternatives to sliced_llama based on common topics and language

llama.cpp

780 58,425 10.0 C++ sliced_llama VS llama.cpp

LLM inference in C/C++
exllamav2

17 3,010 9.8 Python sliced_llama VS exllamav2

A fast inference library for running LLMs locally on modern consumer-class GPUs
InfluxDB

www.influxdata.com featured

Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
magi_llm_gui

4 39 8.7 Python sliced_llama VS magi_llm_gui

A Qt GUI for large language models
dbrx

4 2,421 5.9 Python sliced_llama VS dbrx

Code examples and resources for DBRX, a large language model developed by Databricks
GenerativeAIExamples

1 1,597 7.5 Python sliced_llama VS GenerativeAIExamples

Generative AI reference workflows optimized for accelerated infrastructure and microservice architecture.
llmflows

1 624 8.6 Python sliced_llama VS llmflows

LLMFlows - Simple, Explicit and Transparent LLM Apps
local-llm-function-calling

1 278 7.0 Python sliced_llama VS local-llm-function-calling

A tool for generating function arguments and choosing what function to call with local LLMs
SaaSHub

www.saashub.com featured

SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives

NOTE: The number of mentions on this list indicates mentions on common posts plus user suggested alternatives. Hence, a higher number means a better sliced_llama alternative or higher similarity.

Suggest an alternative to sliced_llama

sliced_llama reviews and mentions

Posts with mentions or reviews of sliced_llama. We have used some of these posts to build our list of alternatives and similar projects. The last one was on 2024-04-08.

Mixture-of-Depths: Dynamically allocating compute in transformers
3 projects | news.ycombinator.com | 8 Apr 2024

There are already some implementations out there which attempt to accomplish this!
Here's an example: https://github.com/silphendio/sliced_llama
A gist pertaining to said example: https://gist.github.com/silphendio/535cd9c1821aa1290aa10d587...
Here's a discussion about integrating this capability with ExLlama: https://github.com/turboderp/exllamav2/pull/275
And same as above but for llama.cpp: https://github.com/ggerganov/llama.cpp/issues/4718#issuecomm...