flake
gpt4all-nix
| flake | gpt4all-nix | |
|---|---|---|
| 5 | 1 | |
| 593 | 37 | |
| 3.9% | - | |
| 4.4 | 10.0 | |
| 7 days ago | 3 months ago | |
| Nix | Nix | |
| GNU Affero General Public License v3.0 | - |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
flake
- Running AI Models on NixOS
- Nixified.Ai Release 2
-
Llama.cpp: Full CUDA GPU Acceleration
> Ideally, there's Nix (and poetry2nix) that could take care of everything, but only a few folks write Flakes for their projects.
Relevant to "AI, Python, setting up is hard ... nix", there's stuff like:
https://github.com/nixified-ai/flake
-
Can you substitute conda with Nix for Data Science and ML/AI?
However, I would reach out to the Nixified.ai folks about it, because I can see that the invoke.ai build script mentions pytorch and several other hard-to-install packages (albeit not detectron).
- A Nix flake for many AI projects
gpt4all-nix
-
Running AI Models on NixOS
I'm (whenever I find the time) maintaining a gpt4all Flake for NixOS: https://github.com/polygon/gpt4all-nix/
What are some alternatives?
nonguix - Nonguix mirror – pull requests ignored, please use upstream for that
text-generation-webui - A Gradio web UI for Large Language Models. Supports transformers, GPTQ, AWQ, EXL2, llama.cpp (GGUF), Llama models.
guix-nonfree - Unofficial collection of packages that are not going to be accepted in to guix
llama.cpp - LLM inference in C/C++
lit-llama - Implementation of the LLaMA language model based on nanoGPT. Supports flash attention, Int8 and GPTQ 4bit quantization, LoRA and LLaMA-Adapter fine-tuning, pre-training. Apache 2.0-licensed.
nix-stable-diffusion - Flake for running SD on NixOS
llama_cpp.rb - llama_cpp provides Ruby bindings for llama.cpp
koboldcpp-rocm - AI Inferencing at the Edge. A simple one-file way to run various GGML models with KoboldAI's UI with AMD ROCm offloading
serving - A flexible, high-performance serving system for machine learning models
guix-nonfree
TokenHawk - WebGPU LLM inference tuned by hand [Moved to: https://github.com/kayvr/token-hawk]
llama.cpp - Port of Facebook's LLaMA model in C/C++