The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning. Learn more →
Top 23 language-model Open-Source Projects
-
-
InfluxDB
Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
-
-
Open-Assistant
OpenAssistant is a chat-based assistant that understands tasks, can interact with third-party systems, and retrieve information dynamically to do so.
-
-
-
WorkOS
The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.
-
-
haystack
:mag: LLM orchestration framework to build customizable, production-ready LLM applications. Connect components (models, vector DBs, file converters) to pipelines or agents that can interact with your data. With advanced retrieval methods, it's best suited for building RAG, question answering, semantic search or conversational agent chatbots.
-
RWKV-LM
RWKV is an RNN with transformer-level LLM performance. It can be directly trained like a GPT (parallelizable). So it's combining the best of RNN and transformer - great performance, fast inference, saves VRAM, fast training, "infinite" ctx_len, and free sentence embedding.
-
-
-
-
LMFlow
An Extensible Toolkit for Finetuning and Inference of Large Foundation Models. Large Models for All.
-
-
-
txtai
💡 All-in-one open-source embeddings database for semantic search, LLM orchestration and language model workflows
-
gpt-neox
An implementation of model parallel autoregressive transformers on GPUs, based on the DeepSpeed library.
-
-
-
-
SaaSHub
SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives
Most of this tutorial is based on Hugging Face course about Transformers and on Niels Rogge's Transformers tutorials: make sure to check their work and give them a star on GitHub, if you please ❤️
Project mention: gpt4-openai-api VS gpt4free - a user suggested alternative | libhunt.com/r/gpt4-openai-api | 2024-01-04I cant install
For open assistant, the code: https://github.com/LAION-AI/Open-Assistant/tree/main/inference
Alpaca is an instruction-oriented LLM derived from LLaMA, enhanced by Stanford researchers with a dataset of 52,000 examples of following instructions, sourced from OpenAI’s InstructGPT through the self-instruct method. The extensive self-instruct dataset, details of data generation, and the model refinement code were publicly disclosed. This model complies with the licensing requirements of its base model. Due to the utilization of InstructGPT for data generation, it also adheres to OpenAI’s usage terms, which prohibit the creation of models competing with OpenAI. This illustrates how dataset restrictions can indirectly affect the resulting fine-tuned model.
Depends what model you want to train, and how well you want your computer to keep working while you're doing it.
If you're interested in large language models there's a table of vram requirements for fine-tuning at [1] which says you could do the most basic type of fine-tuning on a 7B parameter model with 8GB VRAM.
You'll find that training takes quite a long time, and as a lot of the GPU power is going on training, your computer's responsiveness will suffer - even basic things like scrolling in your web browser or changing tabs uses the GPU, after all.
Spend a bit more and you'll probably have a better time.
[1] https://github.com/hiyouga/LLaMA-Factory?tab=readme-ov-file#...
Project mention: The Era of 1-bit LLMs: ternary parameters for cost-effective computing | news.ycombinator.com | 2024-02-28https://github.com/Stability-AI/StableLM?tab=readme-ov-file#...
View on GitHub
https://github.com/BlinkDL/RWKV-LM#rwkv-discord-httpsdiscord... lists a number of implementations of various versions of RWKV.
https://github.com/BlinkDL/RWKV-LM#rwkv-parallelizable-rnn-w... :
> RWKV: Parallelizable RNN with Transformer-level LLM Performance (pronounced as "RwaKuv", from 4 major params: R W K V)
> RWKV is an RNN with Transformer-level LLM performance, which can also be directly trained like a GPT transformer (parallelizable). And it's 100% attention-free. You only need the hidden state at position t to compute the state at position t+1. You can use the "GPT" mode to quickly compute the hidden state for the "RNN" mode.
> So it's combining the best of RNN and transformer - great performance, fast inference, saves VRAM, fast training, "infinite" ctx_len, and free sentence embedding (using the final hidden state).
> "Our latest version is RWKV-6,*
Project mention: DECT NR+: A technical dive into non-cellular 5G | news.ycombinator.com | 2024-04-02This seems to be an order of magnitude better than LoRa (https://lora-alliance.org/ not https://arxiv.org/abs/2106.09685). LoRa doesn't have all the features this one does like OFDM, TDM, FDM, and HARQ. I didn't know there's spectrum dedicated for DECT use.
Hugging Face seems to like Rust. They also wrote Tokenizers in Rust.
While OpenAI’s CLIP model has garnered a lot of attention, it is far from the only game in town—and far from the best! On the OpenCLIP leaderboard, for instance, the largest and most capable CLIP model from OpenAI ranks just 41st(!) in its average zero-shot accuracy across 38 datasets.
Project mention: SpeechBrain 1.0: A free and open-source AI toolkit for all things speech | news.ycombinator.com | 2024-02-28
Project mention: Building a SQL Expert Bot: A Step-by-Step Guide with Vercel AI SDK and OpenAI API | dev.to | 2024-03-05The Vercel AI SDK is built for OpenAI APIs and includes a range of tools for utilizing OpenAI APIs.
txtai is an all-in-one embeddings database for semantic search, LLM orchestration and language model workflows.
CogVLM is very good in my (brief) testing: https://github.com/THUDM/CogVLM
The model weights seem to be under a non-commercial license, not true open source, but it is "open access" as you requested.
The easiest is to use vllm (https://github.com/vllm-project/vllm) to run it on a Couple of A100's, and you can benchmark this using this library (https://github.com/EleutherAI/lm-evaluation-harness)
Project mention: New OS Python Framework "Agents" Introduced for Autonomous Language Agents | /r/deeplearning | 2023-09-21(arXiv) (github)
NOTE:
The open source projects on this list are ordered by number of github stars.
The number of mentions indicates repo mentiontions in the last 12 Months or
since we started tracking (Dec 2020).
The latest post mention was on 2024-04-16.
language-model related posts
- Mistral AI Launches New 8x22B Moe Model
- Schedule-Free Learning – A New Way to Train
- DECT NR+: A technical dive into non-cellular 5G
- Prompt Engineering Guide
- Show HN: UForm v2 Featuring Multimodal Matryoshka, Multimodal DPO, and ONNX
- Show HN: LlamaGym – fine-tune LLM agents with online reinforcement learning
- Show HN: Fructose, LLM calls as strongly typed functions
-
A note from our sponsor - WorkOS
workos.com | 19 Apr 2024
Index
What are some of the best open-source language-model projects? This list will help you:
| Project | Stars | |
|---|---|---|
| 1 | transformers | 124,557 |
| 2 | gpt4free | 56,769 |
| 3 | Prompt-Engineering-Guide | 43,498 |
| 4 | Open-Assistant | 36,601 |
| 5 | stanford_alpaca | 28,723 |
| 6 | mlc-llm | 16,622 |
| 7 | LLaMA-Factory | 16,319 |
| 8 | StableLM | 15,848 |
| 9 | haystack | 13,564 |
| 10 | RWKV-LM | 11,579 |
| 11 | ChatRWKV | 9,266 |
| 12 | LoRA | 8,956 |
| 13 | tokenizers | 8,375 |
| 14 | open_clip | 8,306 |
| 15 | LMFlow | 7,975 |
| 16 | speechbrain | 7,836 |
| 17 | ai | 7,641 |
| 18 | txtai | 6,910 |
| 19 | gpt-neox | 6,556 |
| 20 | BERT-pytorch | 5,979 |
| 21 | CogVLM | 4,896 |
| 22 | lm-evaluation-harness | 4,848 |
| 23 | agents | 4,495 |
SaaSHub - Software Alternatives and Reviews
SaaSHub helps you find the best software and product alternatives
www.saashub.com