airweave VS txtai

I'm working on Airweave https://github.com/airweave-ai/airweave , an open-source dev tool that makes any app searchable for AI agents. it connects to a source app, db, or api and converts its contents to accessible knowledge for agents. Airweave automates authentication, ingestion, enrichment, mapping, and syncing to vector stores and graph databases of choice. you can use it via our UI, API, or SDKs https://docs.airweave.ai/

we originally built this for our previous agent startup as an internal solution to ensure agents could find the relevant data on apps they're using. We then pivoted to this after some early positive reactions and decided to open-source it.

here's a short demo: https://tinyurl.com/demo-airweave

we're two engineers/friends based in Amsterdam, NL. We just launched the project, so it's rough around the edges ofc, but we're very eager to get some feedback!

feel free to reach out to me personally if you like this!

txtai is an all-in-one embeddings database for semantic search, LLM orchestration and language model workflows.

Ideal For: Projects requiring quick setup and robust search capabilities. GitHub Repository

Excellent project.

As mentioned in another comment, I've put together an embeddings database using the arxiv dataset (https://huggingface.co/NeuML/txtai-arxiv) recently.

For those interested in the literature search space, a couple other projects I've worked on that may be of interest.

annotateai (https://github.com/neuml/annotateai) - Annotates papers with LLMs. Supports searching the arxiv database mentioned above.

paperai (https://github.com/neuml/paperai) - Semantic search and workflows for medical/scientific papers. Built on txtai (https://github.com/neuml/txtai)

paperetl (https://github.com/neuml/paperetl) - ETL processes for medical and scientific papers. Supports full PDF docs.

If you're looking for a lightweight open-source framework designed to handle the patterns mentioned in this article: https://github.com/neuml/txtai

Disclaimer: I'm the author of the framework.

I fully agree. Postgres has solved many of the problems that many are re-solving with GenAI related databases.

With txtai (https://github.com/neuml/txtai), I've went all in with Postgres + pgvector. Projects can start small with a SQLite backend then switch the persistence to Postgres. With this, you get all the years of battle-tested production experience from Postgres built-in for free.

VAD is certainly a complex but underappreciated topic. If you like signal processing, FFTs and want to see a similar concept implemented in Python, then check out this code. It's has a fairly well-tuned VAD component built-in.

https://github.com/neuml/txtai/blob/master/src/python/txtai/...

txtai (https://github.com/neuml/txtai) has had inline vectorization since 2020. It supports Transformers, llama.cpp and LLM API services. It also has inline integration with LLM models and a built-in RAG pipeline.

An alternative you can try is txtai (https://github.com/neuml/txtai).

Disclaimer: I'm the primary developer