hyperloglog VS bitmap

For the last 1.5 years, I have been using Axiom for all of my logs ingestion, querying, and monitoring needs. It is a great product and I never had one issue with it in my time using it. Spoiler alert, even today, when it failed it was actually my fault, but let's see what happened.

Axiom

Axiom is an observability backend with OpenTelemetry support. I chose it for my recent project because of its cost, especially for hobby projects or serious projects in its early phase. They provide a JavaScript library that you can use as transport for pino.

Axiom has a great integration with Next.js: https://axiom.co

Assuming you want to track and visualize some metrics, check out https://axiom.co/ . They give you 500 GB ingest, 100 GB storage, 10 hrs query hours per month.

Axiom is a data management and analytics platform that enables organizations to gather, manage, and analyze large volumes of data from various sources. The platform is designed to provide users with a comprehensive view of their data, allowing them to make better-informed business decisions.

Axiom - https://axiom.co/

Have you seen axiom.co? specifically targets DD. Interesting to say the least.

I'm also very curious to know this. HyperLogLog is written in Go:

https://github.com/axiomhq/hyperloglog

I would expect V to be a more natural choice for a port than Zig.

Good question, I think there's many different lessons. To your point about bitmasks, you can imagine that each component (i.e. column) has an array of data and a large bitmap that identifies whether a component is present or not. Had to build a SIMD implementation so you can do and, and not, or and xor operations on millions of components within reasonable amount of time. Interestingly enough, you still need a hashmap or b+tree in case you want to retrieve a component by it's ID instead of an index, but the rest of things can be modeled with bitmap indexes.