SaaSHub helps you find the best software and product alternatives Learn more →
Top 23 question-answering Open-Source Projects
-
haystack
:mag: LLM orchestration framework to build customizable, production-ready LLM applications. Connect components (models, vector DBs, file converters) to pipelines or agents that can interact with your data. With advanced retrieval methods, it's best suited for building RAG, question answering, semantic search or conversational agent chatbots.
-
PaddleNLP
👑 Easy-to-use and powerful NLP and LLM library with 🤗 Awesome model zoo, supporting wide-range of NLP tasks from research to industrial applications, including 🗂Text Classification, 🔍 Neural Search, ❓ Question Answering, ℹ️ Information Extraction, 📄 Document Intelligence, 💌 Sentiment Analysis etc.
-
InfluxDB
Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
-
simpletransformers
Transformers for Information Retrieval, Text Classification, NER, QA, Language Modelling, Language Generation, T5, Multi-Modal, and Conversational AI
-
-
vault-ai
OP Vault ChatGPT: Give ChatGPT long-term memory using the OP Stack (OpenAI + Pinecone Vector Database). Upload your own custom knowledge base files (PDF, txt, epub, etc) using a simple React frontend.
-
-
WorkOS
The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.
-
rust-bert
Rust native ready-to-use NLP pipelines and transformer-based models (BERT, DistilBERT, GPT2,...)
-
FARM
:house_with_garden: Fast & easy transfer learning for NLP. Harvesting language models for the industry. Focus on Question Answering.
-
bootcamp
Dealing with all unstructured data, such as reverse image search, audio search, molecular search, video analysis, question and answer systems, NLP, etc. (by milvus-io)
-
Awesome-LLM-Reasoning
Reasoning in Large Language Models: Papers and Resources, including Chain-of-Thought, Instruction-Tuning and Multimodality.
-
ThoughtSource
A central, open resource for data and tools related to chain-of-thought reasoning in large language models. Developed @ Samwald research group: https://samwald.info/
-
primeqa
The prime repository for state-of-the-art Multilingual Question Answering research and development.
-
Giveme5W1H
Extraction of the journalistic five W and one H questions (5W1H) from news articles: who did what, when, where, why, and how?
-
happy-transformer
Happy Transformer makes it easy to fine-tune and perform inference with NLP Transformer models.
-
lumos
Code and data for "Lumos: Learning Agents with Unified Data, Modular Design, and Open-Source LLMs" (by allenai)
-
PIXIU
This repository introduces PIXIU, an open-source resource featuring the first financial large language models (LLMs), instruction tuning data, and evaluation benchmarks to holistically assess financial LLMs. Our goal is to continually push forward the open-source development of financial artificial intelligence (AI).
-
megabots
🤖 State-of-the-art, production ready LLM apps made mega-easy, so you don't have to build them from scratch 🤯 Create a bot, now 🫵
-
SaaSHub
SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives
View on GitHub
Project mention: Spark NLP 5.1.0: Introducing state-of-the-art OpenAI Whisper speech-to-text, OpenAI Embeddings and Completion transformers, MPNet text embeddings, ONNX support for E5 text embeddings, new multi-lingual BART Zero-Shot text classification, and much more! | /r/Python | 2023-09-06
Project mention: Oracle of Zotero: LLM QA of Your Research Library | news.ycombinator.com | 2023-11-26Why does this post link to a renamed fork of Paper-QA (https://github.com/whitead/paper-qa) which has made zero changes and is 19 commits behind the original?
Project mention: I built an open source website that lets you upload large files, such as in-depth novels/ebooks or academic papers, and ask GPT4 questions based on your specific knowledge base. So far, I've tested it with long books like the Odyssey and random research PDFs, and I'm shocked at how incisive it is. | /r/ChatGPT | 2023-08-05
Project mention: More Agents Is All You Need: LLMs performance scales with the number of agents | news.ycombinator.com | 2024-04-06I couldn't agree more. You should check out LLMWare's SLIM agents (https://github.com/llmware-ai/llmware/tree/main/examples/SLI...). It's focusing on pretty much exactly this and chaining multiple local LLMs together.
A really good topic that ties in with this is the need for deterministic sampling (I may have the terminology a bit incorrect) depending on what the model is indended for. The LLMWare team did a good 2 part video on this here as well (https://www.youtube.com/watch?v=7oMTGhSKuNY)
I think dedicated miniture LLMs are the way forward.
Disclaimer - Not affiliated with them in any way, just think it's a really cool project.
Project mention: How to leverage the state-of-the-art NLP models in Rust | /r/infinilabs | 2023-06-07brew install libtorch brew link libtorch brew ls --verbose libtorch | grep dylib export LIBTORCH=$(brew --cellar pytorch)/$(brew info --json pytorch | jq -r '.[0].installed[0].version') export LD_LIBRARY_PATH=${LIBTORCH}/lib:$LD_LIBRARY_PATH git clone https://github.com/guillaume-be/rust-bert.git cd rust-bert ORT_STRATEGY=system cargo run --example sentence_embeddings
Project mention: Techbro says that GPT models will soon have over 9000 IQ in ~5 years | /r/SneerClub | 2023-05-04
I have tried to do some searching for models but there don't seem to be ones that do what I am looking for. The closest I found was Questgen, but it only generated the questions and they, more often than, not did not make sense - especially for the types of questions I was looking to generate.
Project mention: Show HN: LLMFlows – LangChain alternative for explicit and transparent apps | news.ycombinator.com | 2023-07-29
Project mention: Lumos: Learning Agents with Unified Data, Modular Design, and Open-Source LLMs | news.ycombinator.com | 2024-04-01Guess you are looking for this - https://github.com/allenai/lumos/blob/main/README.md
Project mention: 🤖 Release 0.0.11 in Megabots | Memory and Vectorstores are live! | /r/LLMDevs | 2023-04-26
NOTE:
The open source projects on this list are ordered by number of github stars.
The number of mentions indicates repo mentiontions in the last 12 Months or
since we started tracking (Dec 2020).
The latest post mention was on 2024-04-07.
question-answering related posts
- Generative AI Frameworks and Tools Every Developer Should Know!
- Best way to programmatically extract data from a set of .pdf files?
- Llama2 and Haystack on Colab
- Build with LLMs for production with Haystack – has 10k stars on GitHub
- Show HN: Haystack – Production-Ready LLM Framework
- Creating search engine for your local network - Is it even possible?
- Show HN: "banks" Using Jinja as the basis of LLM prompt templating
-
A note from our sponsor - SaaSHub
www.saashub.com | 19 Apr 2024
Index
What are some of the best open-source question-answering projects? This list will help you:
| Project | Stars | |
|---|---|---|
| 1 | haystack | 13,564 |
| 2 | PaddleNLP | 11,386 |
| 3 | simpletransformers | 3,972 |
| 4 | spark-nlp | 3,667 |
| 5 | paper-qa | 3,593 |
| 6 | vault-ai | 3,217 |
| 7 | llmware | 3,056 |
| 8 | rust-bert | 2,415 |
| 9 | FARM | 1,723 |
| 10 | bootcamp | 1,606 |
| 11 | Awesome-LLM-Reasoning | 1,062 |
| 12 | Questgen.ai | 871 |
| 13 | ThoughtSource | 832 |
| 14 | primeqa | 696 |
| 15 | llmflows | 612 |
| 16 | qagnn | 588 |
| 17 | fastT5 | 539 |
| 18 | Giveme5W1H | 500 |
| 19 | happy-transformer | 497 |
| 20 | lumos | 404 |
| 21 | PIXIU | 393 |
| 22 | LinkBERT | 389 |
| 23 | megabots | 334 |
SaaSHub - Software Alternatives and Reviews
SaaSHub helps you find the best software and product alternatives
www.saashub.com