I'm writing a new vector search SQLite Extension

• sqlite-vec

  3 493 8.0 C

  Work-in-progress vector search SQLite extension that runs anywhere.

Author here — happy to answer any questions! This is more of a "I'm working on a new project" rather than an official release, the extension itself is still a work-in-progress. A link to the project: https://github.com/asg017/sqlite-vec

I have a pretty specific vision of what v0.1.0 of this extension will look like, but it'll take a few more weeks to get there. This blog post was more for letting users of sqlite-vss (a previous vector search SQLite extension I wrote) know what will be coming next. There will be a much bigger release when that is ready.

But in general, I'm super excited to have an easy embeddable vector search alternative! Especially one that runs on all operating system, in WASM, mobile devices, Raspberry Pis, etc. I personally I'm trying to run a lil' semantic search app on my Beepy[0], which is a ton of fun to play with.

[0] https://beepy.sqfmi.com/

• beepy

  9 456 6.9 Shell

  Beepy

  

Author here — happy to answer any questions! This is more of a "I'm working on a new project" rather than an official release, the extension itself is still a work-in-progress. A link to the project: https://github.com/asg017/sqlite-vec

I have a pretty specific vision of what v0.1.0 of this extension will look like, but it'll take a few more weeks to get there. This blog post was more for letting users of sqlite-vss (a previous vector search SQLite extension I wrote) know what will be coming next. There will be a much bigger release when that is ready.

But in general, I'm super excited to have an easy embeddable vector search alternative! Especially one that runs on all operating system, in WASM, mobile devices, Raspberry Pis, etc. I personally I'm trying to run a lil' semantic search app on my Beepy[0], which is a ton of fun to play with.

[0] https://beepy.sqfmi.com/

• InfluxDB

  www.influxdata.com featured

  Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

  

• sqlite-ecosystem

  2 119 4.7 TypeScript

  An overview of all my SQLite extensions, and a roadmap for future extensions and tooling!

I definitely plan to! I have a much larger list of SQLite extensions I've built here: https://github.com/asg017/sqlite-ecosystem

Here's a few other references you may enjoy if you wanna learn more about SQLite extensions:

- The single source file for sqlite-vec: https://github.com/asg017/sqlite-vec/blob/main/sqlite-vec.c

- sqlean, a project from Anton Zhiyanov which is good base of great SQLite extensions: https://github.com/nalgeon/sqlean

- The official SQLite docs: https://www.sqlite.org/loadext.html

- The "hello world" SQLite extension example: https://www.sqlite.org/src/file/ext/misc/rot13.c

• sqlite-plus

  29 3,405 7.4 C

  The ultimate set of SQLite extensions

I definitely plan to! I have a much larger list of SQLite extensions I've built here: https://github.com/asg017/sqlite-ecosystem

Here's a few other references you may enjoy if you wanna learn more about SQLite extensions:

- The single source file for sqlite-vec: https://github.com/asg017/sqlite-vec/blob/main/sqlite-vec.c

- sqlean, a project from Anton Zhiyanov which is good base of great SQLite extensions: https://github.com/nalgeon/sqlean

- The official SQLite docs: https://www.sqlite.org/loadext.html

- The "hello world" SQLite extension example: https://www.sqlite.org/src/file/ext/misc/rot13.c

• usearch

  21 1,730 9.8 C++

  Fast Open-Source Search & Clustering engine × for Vectors & 🔜 Strings × in C++, C, Python, JavaScript, Rust, Java, Objective-C, Swift, C#, GoLang, and Wolfram 🔍

  

Might have a look at this library:

https://github.com/unum-cloud/usearch

It does HNSW and there is a SQLite related project, though not quite the same thing.

• tg

  6 538 6.8 C

  Geometry library for C - Fast point-in-polygon (by tidwall)

Thanks for sharing! I've looked into usearch before, it's really sleek, especially all their language bindings. Though I want sqlite-vec to have total control over what stays in-memory vs on-disk during searches, and most vector search libraries like usearch/hnswlib/faiss/annoy either always store in-memory or don't offer hooks for other storage systems.

Additionally, sqlite-vec takes advantage of some SQLite specific APIs, like BLOB I/O [0], which I hope would speed things up a ton. It's a ton more work, coming up with new storage solutions that are backed by SQLite shadow tables, but I think it'll be work it!

And I also like how sqlite-vec is just a single sqlite-vec.c file. It makes linking + cross-compiling super easy, and since I got burned relying on heavy C++ dependencies with sqlite-vss, a no-dependency rule feels good. Mostly inspired by SQLite single-file sqlite3.c amalgamation, and Josh Baker's single file C projects like tg[1].

[0] https://www.sqlite.org/c3ref/blob_open.html

[1] https://github.com/tidwall/tg

• sqlite-vss

  17 1,518 6.8 C++

  A SQLite extension for efficient vector search, based on Faiss!

I guess this is an answer to the GitHub issue I opened against SQLite-vss a couple of months ago?

https://github.com/asg017/sqlite-vss/issues/124

• SaaSHub

  www.saashub.com featured

  SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives

  

• agentcloud

  7 371 9.9 TypeScript

  Agent Cloud is like having your own GPT builder with a bunch extra goodies. The GUI features 1) RAG pipeline which can natively embed 260+ datasources 2) Create Conversational apps (like GPTs) 3) Create Multi Agent process automation apps (crewai) 4) Tools 5) Teams+user permissions. Get started fast with Docker and our install.sh

  

This is great. we used Qdrant vector DB for an end to end automation for our AI RAG app at https://github.com/rnadigital/agentcloud, but very excited to see you are making a successor. any ETA when it would be ready to use and any quickstart guide? Maybe I can help in writing a blog as well.

• cozo

  30 3,137 8.0 Rust

  A transactional, relational-graph-vector database that uses Datalog for query. The hippocampus for AI!

  

Any thoughts on how your project will compare to CozoDB?

https://github.com/cozodb/cozo

• portable-hnsw

  2 77 7.3 HTML

  What if an HNSW index was just a file, and you could serve it from a CDN, and search it directly in the browser?

This is exciting!

This could really simplify a little CDN-based HNSW project I did (https://github.com/jasonjmcghee/portable-hnsw)

Seems like with duckdb vss you could just embed, store as duckdb format, then execute sql against it (in the CDN).

• dwata

  1 107 9.2 TypeScript

  AI studio for you and your business. Create assistants, connect databases, APIs (like Stripe) or CSV/Excel files. Use AI to create insights, workflows, action items.

I have been looking at your extension for the last couple weeks and I think I will end up using it. I am also looking at running qdrant locally. I am creating an open source AI studio (desktop app) (1). It is in very early stages but it is fun to get to know this landscape. I would be happy to contribute to your project in any way I can. And thank you for this project.

1. https://github.com/brainless/dwata

Suggest a related project