Our great sponsors
-
libvineyard
Discontinued vineyard (v6d): an in-memory immutable data manager. [Moved to: https://github.com/alibaba/v6d]
-
feather
Feather: fast, interoperable binary data frame storage for Python, R, and more powered by Apache Arrow (by wesm)
-
InfluxDB
Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
6. Kubernetes-integration for large-scale big data applications
Github: https://github.com/alibaba/libvineyard (s are welcomed!)
It'd be interesting to know how this compares with alternative solutions.
I might not understand the benefit proposition correctly, and I'm not specifically into Python for data work, but I immediately thought of things like feather[1], fst[2], disk.frame[3] and even DuckDB[4].
Some of these are on disk rather than in memory, but I'd still be interested in performance and use case comparisons.
[1] https://github.com/wesm/feather
[2] https://www.fstpackage.org/fst/
[3] https://diskframe.com/
[4] https://duckdb.org/