ggml
chat.petals.dev
| ggml | chat.petals.dev | |
|---|---|---|
| 3 | 8 | |
| 19 | 299 | |
| - | 2.7% | |
| 8.6 | 7.1 | |
| 7 months ago | about 1 month ago | |
| Python | ||
| MIT License | - |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
ggml
-
Run LLMs at home, BitTorrent‑style
https://github.com/philpax/ggml/blob/gguf-spec/docs/gguf.md#...
It is (IMO) a necessary and good change.
I just specified gguf because my 3090 cannot host a 70B model without offloading outside of exLlama's very new ~2 bit quantization.
- GGUF File Format Specification
-
Meta: Code Llama, an AI Tool for Coding
While we're at it, the GGML file format has been deprecated in favor of GGUF.
https://github.com/philpax/ggml/blob/gguf-spec/docs/gguf.md
https://github.com/ggerganov/llama.cpp/pull/2398
chat.petals.dev
-
Make no mistake—AI is owned by Big Tech
ETA: https://chat.petals.dev
-
Run LLMs in bittorrent style
Check it out at Petals.dev. Chatbot
-
Run LLMs at home, BitTorrent‑style
Hi, a dev here. means "end of sequence" for LLMs. If a model generates it, it forgets everything and continue with an unrelated random text. So I don't think that malicious actors are involved here.
Apparently, the Colab code snippet is just too simplified and does not handle correctly. This is not the case with the full chatbot app at https://chat.petals.dev - you can use it instead or take a look at its code.
-
Falcon180B: authors open source a new 180B version!
edit: this community of people is amazing. like 10 minutes after I posted this or so.... it is now up on chat.petals.dev !!!!
- Talk to Falcon 180B-Chat running over Petals
-
ChatGPT Is Down Again
good opportunity to try the free and totally open source Big Science Petals chat: https://chat.petals.dev/ ... Try out Stable Beluga 2 70B
I am currently running my 3090 GPU on there to help out, you can check out https://health.petals.dev/
If you have a spare GPU, consider contributing: https://github.com/bigscience-workshop/petals . I am not associated with them.
-
Sweating Bullets Test
So far, not a single one of the models tested (between 7b-70b) could figure out the name of the main character (Nick Slaughter). I've tried all sorts of prompts and the connection between "Tropical Heat" and "Sweating Bullets" is usually known to the model (e.g. "What's the show "Tropical Heat" called in the US?"). But as soon as I ask about the main character, all the models I have tested so far hallucinate all sorts of names, though usually in the right direction (detectives).
- Petals: Run 100B+ models at home bit-torrent style
What are some alternatives?
ollama - Get up and running with Llama 3, Mistral, Gemma, and other large language models.
askai - Command Line Interface for OpenAi ChatGPT
smartcat
petals - 🌸 Run LLMs at home, BitTorrent-style. Fine-tuning and inference up to 10x faster than offloading
LoRA - Code for loralib, an implementation of "LoRA: Low-Rank Adaptation of Large Language Models"
godot-dodo - Finetuning large language models for GDScript generation.
KoboldAI-Client
codellama - Inference code for CodeLlama models
ollama-ui - Simple HTML UI for Ollama
artbot-for-stable-diffusion - A front-end GUI for interacting with the AI Horde / Stable Diffusion distributed cluster