Our great sponsors
-
petals
🌸 Run LLMs at home, BitTorrent-style. Fine-tuning and inference up to 10x faster than offloading
-
WorkOS
The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.
https://github.com/bigscience-workshop/petals
Inference only: https://lite.koboldai.net/
If you want to host for the AI Horde, Aphrodite has the most GPU throughput, and Koboldcpp is the easiest and quite flexible:
https://github.com/PygmalionAI/aphrodite-engine
https://github.com/LostRuins/koboldcpp
I havent messed with Petals yet either, but swarm finetuning is very appealing... If thats even implemented atm.
NOTE:
The number of mentions on this list indicates mentions on common posts plus user suggested alternatives.
Hence, a higher number means a more popular project.
Related posts
- Creando SubtĂtulos Automáticos para VĂdeos con Python, Faster-Whisper, FFmpeg, Streamlit, Pillow
- SatCLIP: A Global, General-Purpose Geographic Location Encoder
- Haystack DB – 10x faster than FAISS with binary embeddings by default
- PySheets – Spreadsheet UI for Python
- AWS Serverless Diversity: Multi-Language Strategies for Optimal Solutions