llama2-high-level-cpp
Inference Llama2 with High-Level C++. (by frost-beta)
PowerInfer
High-speed Large Language Model Serving on PCs with Consumer-grade GPUs (by SJTU-IPADS)
| llama2-high-level-cpp | PowerInfer | |
|---|---|---|
| 1 | 4 | |
| 7 | 6,996 | |
| - | 3.5% | |
| 3.8 | 9.8 | |
| 3 months ago | 17 days ago | |
| C | C++ | |
| MIT License | MIT License |
The number of mentions indicates the total number of mentions that we've tracked plus the number of user suggested alternatives.
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
llama2-high-level-cpp
Posts with mentions or reviews of llama2-high-level-cpp.
We have used some of these posts to build our list of alternatives
and similar projects.
PowerInfer
Posts with mentions or reviews of PowerInfer.
We have used some of these posts to build our list of alternatives
and similar projects. The last one was on 2023-12-26.
- FLaNK 25 December 2023
- High-Speed Large Language Model Serving on PCs with Consumer-Grade GPUs
-
PowerInfer: Fast Large Language Model Serving with a Consumer-Grade GPU [pdf]
> PowerInfer’s source code is publicly available at https://github.com/SJTU-IPADS/PowerInfer
- PowerInfer: High-Speed Large Language Model Serving on Consumer-Grade GPUs
What are some alternatives?
When comparing llama2-high-level-cpp and PowerInfer you can also consider the following projects:
Cgml - GPU-targeted vendor-agnostic AI library for Windows, and Mistral model implementation.
zilla-examples - A collection of pre-canned Zilla feature demos. Deploy on K8s via Helm.
llama.cpp - LLM inference in C/C++
GenerativeAIExamples - Generative AI reference workflows optimized for accelerated infrastructure and microservice architecture.
FLaNK-SaoPauloBrazil - FLaNK-SaoPauloBrazil
k3s - Lightweight Kubernetes
mlx - MLX: An array framework for Apple silicon
direnv - unclutter your .profile