towhee
sqlite-vss
towhee | sqlite-vss | |
---|---|---|
26 | 16 | |
2,989 | 1,455 | |
2.4% | - | |
8.6 | 8.0 | |
3 months ago | about 2 months ago | |
Python | C++ | |
Apache License 2.0 | MIT License |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
towhee
- FLaNK Stack Weekly for 14 Aug 2023
- Welcome to generate your embeddings with Towhee
-
Vector database built for scalable similarity search
As another commenter noted, Milvus is overkill and a "bit much" if you're learning/playing.
A good intro to the field with progression towards a full Milvus implementation could be starting with towhee[0] (which is also supported by Milvus).
towhee has an example to do exactly what you want with CLIP[1].
[0] - https://towhee.io/
[1] - https://github.com/towhee-io/examples/tree/main/image/text_i...
-
What Is DocArray?
The description of this is kind of confusing but I think the easiest way to understand it is that it is a data processing pipeline of sorts. Take unstructured data and apply transformation and computation. A similar project to this is Towhee (https://github.com/towhee-io/towhee). This project tries to simplify unstructured data processing and provides pretrained models and pipelines from their hub.
-
[P] My co-founder and I quit our engineering jobs at AWS to build “Tensor Search”. Here is why.
Milvus also has incredible flexibility when it comes to choosing an indexing strategy, and we also have a library specifically meant to help vectorize a variety of data called Towhee (https://github.com/towhee-io/towhee).
-
Deep Dive into Real-World Image Search Engine with Python
Benchmarking the models with towhee is as simple as:
-
A quick tip on DataFrame.apply
The project's homepage is https://github.com/towhee-io/towhee, and you can find more about towhee by going through the documents.
-
Build an Image Search Engine in Minutes
I made a tutorial for building an image search engine with python. The code example is as simple as 10 lines of code, using Towhee and Milvus To put images into the search engine:
-
Any good libraries for feature extraction?
Traditionally, I've done this through PyTorch by adding a hook, but this requires knowledge of the model itself (i.e. model arch and layer names). I found https://github.com/Hironsan/awesome-embedding-models but it didn't provide many CV-focused open-source projects. There's also https://github.com/towhee-io/towhee which is great but more targeted towards application development.
-
A python framework for unstructured data processing
You can check the result from the tutorial.
sqlite-vss
-
Embeddings are a good starting point for the AI curious app developer
Perhaps sqlite-vss? It adds vector searches to sqlite.
https://github.com/asg017/sqlite-vss
-
How to Enhance Content with Semantify
Utilizing sqlite-vss to store and query vector embeddings managed by a local SQLite database, Semantify conducts fast, precise vector searches within these embeddings to find and recommend relevant content, ensuring readers are presented with articles that truly match their interests.
-
SQLite vs. Chroma: A Comparative Analysis for Managing Vector Embeddings
Whether you’re navigating through well-known options like SQLite, enriched with the sqlite-vss extension, or exploring other avenues like Chroma, an open-source vector database, selecting the right tool is paramount. This article compares these two choices, guiding you through the pros and cons of each, helping you choose the right tool for storing and querying vector embeddings for your project.
-
Vector database is not a separate database category
Here is a SQLite extension that uses Faiss under the hood.
https://github.com/asg017/sqlite-vss
Not associated with the project, just love SQLite and find it very useful.
- SQLite-Vss: A SQLite Extension for Vector Search
-
Introduction to Vector Search and Embeddings
Vector Databases: As your data grows, efficiently searching through millions of vectors can become a challenge. Specialized vector databases like FAISS, Annoy, or Elasticsearch's vector search capabilities can be explored to manage and search through large-scale vector data. Your sentence is grammatically correct. In addition, databases like SQLite and PostgreSQL have extensions, such as sqlite-vss and pgvector, that can be used to store and query vector embeddings, respectively.
-
The Problem with LangChain
I had a go at one of those a few months ago: https://datasette.io/plugins/datasette-faiss
Alex Garcia built a better one here as a SQLite Rust extension: https://github.com/asg017/sqlite-vss
-
Every request, every microsecond: scalable machine learning at Cloudflare
Since the problem domain is that of anomaly detection from constructed request feature embeddings, I wonder if an ANN-search methodology using an embedded database (such as https://github.com/asg017/sqlite-vss or similar) was explored.
-
Disrupting the AI Scene with Open Source and Open Innovation
As I searched for "sqlite vector plugin" I didn't find any results, before a couple of weeks ago. Two weeks ago I found Alex' SQLite VSS plugin for SQLite. The library was an amazing piece of engineering from an "idea perspective". However, as I started playing around with it, I realised it was ipso facto like "Titanic". Beautiful and amazing, but destined to leak water and sink to the bottom of the ocean because of what we software engineers refers to as "memory leaks".
- SQLite extension for efficient vector search, based on Faiss
What are some alternatives?
qdrant - Qdrant - High-performance, massive-scale Vector Database for the next generation of AI. Also available in the cloud https://cloud.qdrant.io/
semantic-kernel - Integrate cutting-edge LLM technology quickly and easily into your apps
Milvus - A cloud-native vector database, storage for next generation AI applications
chroma - the AI-native open-source embedding database
examples - Analyze the unstructured data with Towhee, such as reverse image search, reverse video search, audio classification, question and answer systems, molecular search, etc.
pgvector-go - pgvector support for Go
PySceneDetect - :movie_camera: Python and OpenCV-based scene cut/transition detection program & library.
milvus-lite - A lightweight version of Milvus wrapped with Python.
AI - Artificial Intelligence Projects
typesense-instantsearch-semantic-search-demo - A demo that shows how to build a semantic search experience with Typesense's vector search feature and Instantsearch.js
pgvector - Open-source vector similarity search for Postgres
txtai - 💡 All-in-one open-source embeddings database for semantic search, LLM orchestration and language model workflows