bbolt VS Weaviate

Based on this [GitHub document](https://github.com/etcd-io/bbolt) for BBoltDB, we can understand that Go Code be used to create a BBoltDB database on the system. The key-values added & operations done on them in that Go Code are stored in the BBoltDB database.

A Locker is a store on your file system (built on top of the amazing bbolt).

- bbolt for storage on disk. In order to get the smallest db file size possible make sure you insert the keys in order and set:

I'm wanting to store some wasm modules (as []byte) in BoltDB. Right now the modules are <1MB, but eventually, they could be 10-50MB in size. Is this going to reduce the performance of BoltDB all around, if the size of a value is this large? If it makes a difference, I'm using the Storm toolkit for querying.

bbolt - An embedded key/value database for Go.

For learning goroutines and channels, I usually recommend writing a program that reads from files and writes the data in a dummy database with something like https://github.com/etcd-io/bbolt. It's relatively simple and you're more likely to run into common manifestations of concurrency issues running disk operations.

If you would like to explore usage of channels, I highly recommend writing a program that reads from files and writes the data in a dummy database with something like https://github.com/etcd-io/bbolt.

No transactions, no consistency guarantees, no benchmarks, global locks in the storage implementation, a collection is copied in its entirety on every insertion to it...I realize it's not for the same use case as MySQL or MongoDB, but a more obvious comparison here is e.g. https://github.com/etcd-io/bbolt. So why should someone use this over bbolt?

use go-sqlite3 to work with sqlite3 is one choice.

https://github.com/etcd-io/bbolt is another pure go option.

cznic seems like an alternative to bbolt. nice to have some options.

In this tutorial, I introduce Weaviate, an open-source vector database, with the thenlper/gte-base embedding model from Alibaba, through Hugging Face's transformers library.

This will be solved in Weaviate https://github.com/weaviate/weaviate/issues/2424

Link to GitHub -->

Hey @rvrs, I work on Weaviate and we are doing some improvements around increasing write throughput:

1. gRPC. Using gRPC to write vectors has had a really nice performance boost. It is released in Weaviate core but here is still some work on do on the clients. Feel free to get in contact if you would like to try it out.

2. Parameter tuning. lowering `efConstruction` can speed up imports.

3. We are also working on async indexing https://github.com/weaviate/weaviate/issues/3463 which will further speed things up.

In comparison with pgvector, Weaviate has more flexible query options such as hybrid search and quantization to save memory on larger datasets.