Our great sponsors
-
InfluxDB
Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
Data warehouses are columnar stores. They are very different from row-oriented databases - like Postgres, MySQL. Operations on columns - e.g., aggregations (mean of a column) are very efficient.
Most vector databases use one of a few different vector indexing libraries - FAISS, hnswlib, and scann (google only) are popular. The newer vector dbs, like weaviate, have introduced their own indexes, but i haven't seen any performance difference -
Reference: https://ann-benchmarks.com/
Here is a SQLite extension that uses Faiss under the hood.
https://github.com/asg017/sqlite-vss
Not associated with the project, just love SQLite and find it very useful.
As a lover of array languages, I remember being excited to read a futurist article on vector processors and programming languages. It was written right before Wes McKinney worked on Pandas (the J programming language influenced him), and I thought J/APL or another array language was going to explode. J has Jd, in which J is fully integrated. This did not come to pass (yet). No matter, I still enjoy array languages anyway. There's a new array language, uiua[1], that is a mix of array and stack concepts with a good standard library including audio and graphics.
[1] https://www.uiua.org/