-
InfluxDB
Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
> It irks me a lot that you can just do JSON.parse, declare any type on it and call it a day
You should look at zod [0], which validates data with inferred types based on the validation having passed, so you do have the guarantee of the type being correct at runtime.
[0] https://zod.dev
I built this decision tree (LightGBM) compiler last summer: https://github.com/siboehm/lleaves
It get's you ~10x speedups for batch predictions, more if your model is big. It's not complicated, it ended up being <1K lines of Python code. I heard a couple of stories like yours, where people had multi-node spark clusters running LightGBM, and it always amused me because by if you compiled the trees instead you could get rid of the whole cluster.
Wow, very interesting, thanks for this. Daily batch predictions is all we do. I’m the maintainer of miceforest[1], do you think this would integrate well into the package at a brief glance? I’m always looking for ways to make this package faster.
[1] https://github.com/AnotherSamWilson/miceforest