ultraviolet
llama.cpp
Our great sponsors
ultraviolet | llama.cpp | |
---|---|---|
4 | 769 | |
727 | 56,891 | |
- | - | |
5.6 | 10.0 | |
23 days ago | about 17 hours ago | |
Rust | C++ | |
- | MIT License |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
ultraviolet
-
Rapier is a set of 2D and 3D physics engines written in Rust
This one has some more traction: https://github.com/fu5ha/ultraviolet
-
Where can I learn more about SIMD, CPU intrinsics and the like in the context of Rust?
Some other good places to look would be the source code for the ultraviolet crate and for wide.
-
Programming languages in 25 days, Part 2: Reflections on language design
2. Using macros for codegen, as seen in almost all other linear algebra crates, e.g. ultraviolet (https://crates.io/crates/ultraviolet)
The drawback of (1) is that it's hard to understand or extend, and the type errors are insane, and put Boost to shame. The drawback of (2) is that's just a macro. Of course (1) and (2) are not interoperable.
C++ templates are really underappreciated, because they are a single mechanism that gives you both powerful generics and macro-like codegen, while being easy to adopt gradually. (E.g. you can add concepts when it gets unwieldy, but you don't have to rewrite the whole thing.)
- [Media] I finished my first rust project: a path tracer
llama.cpp
-
Phi-3 Weights Released
well https://github.com/ggerganov/llama.cpp/issues/6849
- Lossless Acceleration of LLM via Adaptive N-Gram Parallel Decoding
- Llama.cpp Working on Support for Llama3
-
Embeddings are a good starting point for the AI curious app developer
Have just done this recently for local chat with pdf feature in https://recurse.chat. (It's a macOS app that has built-in llama.cpp server and local vector database)
Running an embedding server locally is pretty straightforward:
- Get llama.cpp release binary: https://github.com/ggerganov/llama.cpp/releases
- Mixtral 8x22B
- Llama.cpp: Improve CPU prompt eval speed
-
Ollama 0.1.32: WizardLM 2, Mixtral 8x22B, macOS CPU/GPU model split
Ah, thanks for this! I can't edit my parent comment that you replied to any longer unfortunately.
As I said, I only compared the contributors graphs [0] and checked for overlaps. But those apparently only go back about year and only list at most 100 contributors ranked by number of commits.
[0]: https://github.com/ollama/ollama/graphs/contributors and https://github.com/ggerganov/llama.cpp/graphs/contributors
-
KodiBot - Local Chatbot App for Desktop
KodiBot is a desktop app that enables users to run their own AI chat assistants locally and offline on Windows, Mac, and Linux operating systems. KodiBot is a standalone app and does not require an internet connection or additional dependencies to run local chat assistants. It supports both Llama.cpp compatible models and OpenAI API.
-
Mixture-of-Depths: Dynamically allocating compute in transformers
There are already some implementations out there which attempt to accomplish this!
Here's an example: https://github.com/silphendio/sliced_llama
A gist pertaining to said example: https://gist.github.com/silphendio/535cd9c1821aa1290aa10d587...
Here's a discussion about integrating this capability with ExLlama: https://github.com/turboderp/exllamav2/pull/275
And same as above but for llama.cpp: https://github.com/ggerganov/llama.cpp/issues/4718#issuecomm...
-
The lifecycle of a code AI completion
For those who might not be aware of this, there is also an open source project on GitHub called "Twinny" which is an offline Visual Studio Code plugin equivalent to Copilot: https://github.com/rjmacarthy/twinny
It can be used with a number of local model services. Currently for my setup on a NVIDIA 4090, I'm running both the base and instruct model for deepseek-coder 6.7b using 5_K_M Quantization GGUF files (for performance) through llama.cpp "server" where the base model is for completions and the instruct model for chat interactions.
llama.cpp: https://github.com/ggerganov/llama.cpp/
deepseek-coder 6.7b base GGUF files: https://huggingface.co/TheBloke/deepseek-coder-6.7B-base-GGU...
deepseek-coder 6.7b instruct GGUF files: https://huggingface.co/TheBloke/deepseek-coder-6.7B-instruct...
What are some alternatives?
Holy-Unblocker - Holy Unblocker is a web proxy service that helps you access websites that may be blocked by your network or browser. It does this securely and with additional features.
ollama - Get up and running with Llama 3, Mistral, Gemma, and other large language models.
30-seconds-of-code - Short code snippets for all your development needs
gpt4all - gpt4all: run open-source LLMs anywhere
2captcha-php - PHP package for easy integration with the API of 2captcha captcha solving service to bypass recaptcha, hcaptcha, funcaptcha, geetest and solve any other captchas.
text-generation-webui - A Gradio web UI for Large Language Models. Supports transformers, GPTQ, AWQ, EXL2, llama.cpp (GGUF), Llama models.
stealth - :rocket: Stealth - Secure, Peer-to-Peer, Private and Automateable Web Browser/Scraper/Proxy [Moved to: https://github.com/tholian-network/stealth]
GPTQ-for-LLaMa - 4 bits quantization of LLaMA using GPTQ
es6-cheatsheet - ES2015 [ES6] cheatsheet containing tips, tricks, best practices and code snippets
ggml - Tensor library for machine learning
33-js-concepts - 📜 33 JavaScript concepts every developer should know.
alpaca.cpp - Locally run an Instruction-Tuned Chat-Style LLM