logs-benchmark VS shite

I am a maintainer at SigNoz. Nice to see OpenObserve's belief that the future of Observability should be OpenSource. We chose clickhouse rather than building a database as it takes multi-year effort to move a db to maturity and clickhouse has been battle-tested at Yandex, Uber and Cloudflare. Clickhouse also provides native integration with s3 and other blob storages. Our users have been using disk as hot storage for a week and moving the data to s3 after that. Tiered storage is really cool in terms of query performance.

We have also seen logs data of our users at a compression ratio of 30x/40x. We have published a logs benchmark (https://github.com/SigNoz/logs-benchmark) where the data is very high-cardinal (causing a compression factor of only 2.5x). Would love to see how does OpenObserve perform in that dataset someday.

Wishing you best for the journey ahead.

Did you update the benchmark after you got feedback from this user at Hacker News or after you got the feedback that including Loki here is kinda pointless?

Pandoc can be your friend. My site maker [1] is built around it.

I think a hundred or so well-chosen lines of your favourite scripting language can do wonders. Mine is ~300 lines of Bash because I over-engineered a thing or two for kicks. The core of it is maybe 50 lines.

[1] https://github.com/adityaathalye/shite

The README documents the architecture and rationale. Maybe it will help you figure out yours. Happy hacking!

[1] https://evalapply.org

[2] https://github.com/adityaathalye/shite?tab=readme-ov-file#te...

Bookmarked! These look like amazing study projects; the kind one can copy and learn from. Quite like how they do it in art school. Each one of them looks like it solves a nontrivial problem, and edifies the reader on the basic contours/tenets of the problem/solution space.

I love this kind of stuff, because it shows one _can_ solve a pretty juicy problem with not that much code, honestly. Also because it suggests that the industrial-strength equivalent has a lot more in for use cases, corner cases, and/or optimisations that are not relevant for one's requirements (at least not yet, maybe not ever).

I aspire to write code like that. Useful, concise, but not obtuse. Some of my code is not as significant as those examples, and maybe falls short of my ideals, but it gets a lot done in well under 500 loc. e.g. my website maker in Bash [1] (hot-builds and hot-refreshes without JS), or the JS that drives text art animations for Hanukkah of Data [2].

[1] https://github.com/adityaathalye/shite is about 350 LoC counted this way (excluding the script containing HTML templates).

  $ grep -E -v "^$|\\s?#" bin/{events,metadata,templating,utils,hotreload}.sh | wc -l

I love the code [1]. Mine [2] is a bit over engineered because I wanted hot-reloading (without JS), and it was a delightful yak shave.

But the basic idea is the same --- heredocs for templating, using a plaintext -> html compiler (pandoc in my case), an intermediate CSV for index generation.

Very nice!

[1] https://github.com/karlb/karl.berlin/blob/master/blog.sh

[2] https://github.com/adityaathalye/shite

xdotool emulates user actions under the X Window System (e.g. typing, mouse around, click etc.).

I'm using it to send keypresses to the browser, as you rightly observe.

So if I want to just reload a page, the browser gets F5.

To GOTO some page, it gets a stream of keystrokes for the URL characters and then Enter.

It's really that simple-minded, and it works!

This case statement covers my usage: https://github.com/adityaathalye/shite/blob/master/bin/hotre...

Pandoc powers my little static site maker:

cf. https://github.com/adityaathalye/shite/blob/master/bin/templ...

  __shite_templating_compile_source_to_html() {