-
-
InfluxDB
Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
-
clj-mmap
A Clojure library designed to allow you to easily mmap files via Java's NIO, and to handle files larger than 2GB.
Thanks for the comment, Why I emphasis at the topic memory mapping, these two project using-memory-mapped-file-huge and iota . Mapping existing java/Clojure data structure with mmap, it may not be the same totally. However, the mechanism behind the coding is quite similar.
Since it's encoded as edn, you could use clojure.edn/read with a file stream and read from the file as in https://github.com/clojure-cookbook/clojure-cookbook/blob/master/04_local-io/4-14_read-write-clojure-data-structures.asciidoc . The edn-seq function defined at https://stackoverflow.com/questions/15234880/how-to-use-clojure-edn-read-to-get-a-sequence-of-objects-in-a-file is also viable if you are lazily reading and processing using sequences.
We have already study cli-mmap & iota, both projects are excellent to house data large than memory heap. We are studying this durable-atom, It is a drop-in replacement for regular clojure atom, uses slurp & spit for reading/writing to the disk. The only drawback is a limit on big data-structures that can't be retained in memory. We are not very experience with the internal clojure but we try to replace slurp & spit with memory mapped file.