hyperdx VS endbasic

Hi HN, Mike and Warren here! We’ve been building an open source local-dev-friendly mode for HyperDX (hyperdx.io). It's a single Docker container that lets you view logs, metrics, and traces for local development in a similar way you can use them for production (live tail, filter/correlate logs & spans, build charts, flamegraph, etc.) Basically, imagine you can run Datadog in a single container for local development. We does this by spinning up an OpenTelemetry collector, Clickhouse DB and HyperDX UI all in a single container - it only takes a few seconds to start and lets you start live tailing your local logs and traces immediately (and graph metrics of course).

README (w/ demo gif): https://github.com/hyperdxio/hyperdx/blob/main/LOCAL.md

We started building local mode as it became one of our most loved [1] community issue after sharing HyperDX. It made sense - as we constantly use HyperDX to debug our own issues locally, and have found it to be a huge productivity boost for things like…

1. Tailing multiple container logs (and grepping/isolating them) without a ton of different terminal splits open at the same time.

2. Be able to sensibly view structured logs and correlated traces to debug local issues (because you’re using structured logging and tracing right?) instead of adding ad-hoc console statements that print out exactly what I could’ve gotten from tracing.

3. Testing telemetry are actually emitting/correlating as expected, before shipping it all the way out to prod and realizing I accidentally created a very expensive high-cardinality metric or a span that has the wrong properties.

We spent some time packaging & tuning our existing OSS stack to run in a single container with less memory/space requirements by staring at `dive` to slim down the image and applied incantations from documentation until memory usage improved. Additionally we removed a few non-local-friendly things like authentication requirements and extraneous services.

It still has all the goodness you’d want - so you can full text search your logs/traces, live tail all your events, view spans correlated with logs (and vice versa), create dashboards based on logs, metrics, traces, and is fully OpenTelemetry compatible - just point your Otel SDK/collector to http://localhost:4318 (or 4317 for the grpc folks) and you’re already good to go.

I’m excited to share what we’ve been working on and would love to hear your feedback and opinions!

Spin up the container yourself to try it out:

docker run -p 8000:8000 -p 4318:4318 -p 4317:4317 -p 8080:8080 -p 8002:8002 hyperdx/hyperdx-local

Main Open Source Repo: https://github.com/hyperdxio/hyperdx

Hosted Demo (in case you want to play around in a cloud sandbox instead): https://api.hyperdx.io/login/demo

HyperDX Landing Page: https://hyperdx.io

[1]: https://github.com/hyperdxio/hyperdx/issues/7

ClickHouse also excels at storing and querying semi-structured data, like event logs. Previously, many engineering teams used Elasticsearch in a similar niche to ClickHouse, building applications like Kibana. Increasingly, developers are choosing ClickHouse over Elasticsearch for its unparalleled performance characteristics. For example, our friends at hyperdx.io are using ClickHouse to build an open-source OpenTelemetry provider!

HyperDX

Biased as a founder in the space [1] but I think with OpenTelemetry + OSS extensible observability tooling, the holy grail of one tool is more realizable than ever.

Vendor lock in with Otel now is hopefully a thing of the past - but now that more obs solutions are going open source, hopefully it's not necessarily true that one tool would be mediocre over all use cases (since DD and the likes are inherently limited by their own engineering teams, vs OSS products can have community/customer contributions to improve the surface area over time on top of the core maintainer's work).

[1] https://github.com/hyperdxio/hyperdx

HyperDX (YC S22) | Founding Engineer | SF Bay Area HQ | REMOTE

We're building an open source, dev friendly observability tool (think Datadog, but something developers actually love to use and companies can actually afford).

We're in the intersection of needing to build rock solid infrastructure ingesting TBs of data, searching it incredibly quickly and scalably, and layering on top a buttery smooth DX from our language-specific SDKs, APIs and web app.

We're super early and hiring our first founding engineer. We already have a cloud product customers pay for and love, loads of runway regardless of the wider economy, 5k+ Github stars weeks after our OSS launch, and tons of hard technical problems.

The vast majority of our work is open source, so you can get a sense of what you'd be working with here: https://github.com/hyperdxio/hyperdx

Our job listing is here as well: https://www.ycombinator.com/companies/hyperdx/jobs/zFXTbzl-f...

Come by our discord as well if you just want to talk shop: https://discord.gg/FErRRKU78j

I'm Mike, one of the cofounders. If you love shipping quickly and want to help us build from the ground up an open source developer tool (that devs won't hate when they're on-call), let me know! mike [at] hyperdx.io

Oh huh! What would you do with span start independent of stops?

A tangent on logcat - local observability to me is a really intriguing area, I think there's a story of Otel for local as well if someone can build a good enough local DX for consuming them (we've been told a number of times about this https://github.com/hyperdxio/hyperdx/issues/7 as an example)

Hi HN, Mike and Warren here! We’ve been building HyperDX (hyperdx.io). HyperDX allows you to easily search and correlate logs, traces, metrics (alpha), and session replays all in one place. For example, if a user reports a bug “this button doesn’t work," an engineer can play back what the user was doing in their browser and trace API calls back to the backend logs for that specific request, all from a single view. Github Repo: https://github.com/hyperdxio/hyperdx

We're building the OSS equivalent when it comes to the observability side of Splunk/DD, on Clickhouse naturally of course but believe in the same end goal of lowering cost via separation of compute and storage.

https://github.com/hyperdxio/hyperdx

I can confirm that writing a terminal is fun, for the reasons mentioned in the article: it’s easy to get “self-hosting”, but then the possibilities are endless :)

In my case, this was about creating the terminal for EndBASIC (https://www.endbasic.dev/). I wanted to mix text and graphics in the same console, so I had to ditch Xterm.js and create my own thing. It was really exciting to see graphics rendering mix with text “just fine” when I was able to render the first line.

I tried to set up a Raspberry Pi and configured it to boot into a simple window manager with DosBox full screen by default. I taught my kids to launch games within that and they learned the very basics… but it didn’t stick: they haven’t really gained any interest in how to do other stuff in the shell.

Anyway: check (my own) https://www.endbasic.dev/ which I’ve written precisely for the situation you describe :) You would actually have to /write/ the games first though!

Slightly buried: Apache 2.0, written in Rust, https://github.com/endbasic/endbasic/

Definitely an interesting attempt to cut through layers of abstraction and make something that lets people make the computer do useful/interesting things. No idea how well they realize that vision, of course, but good idea.

Or you could use https://www.endbasic.dev/

Somebody else brought it up in a separate comment, but because you specifically ask about the Raspberry, I'll mention EndBASIC (https://www.endbasic.dev/) here again :)

Supporting this platform has been a primary goal of mine, and in fact, one of the features (GPIO) only works on the Raspberry Pi today :) But there is a long road ahead. My vision is to create a minimal Linux image that boots straight into EndBASIC, and extend EndBASIC to give you more control of the Pi's hardware. The idea is to truly mimic the old C64 experience, but leveraging the power of modern hardware / infrastructure.

Agree with the author’s thesis of how the folks that “grew with computers” have an advantage over those approaching them now, in terms of understanding the inner workings. I’m not sure that this matters much in terms of solving actual problems though, which is probably a good thing.

But I somehow find it a little bit sad that this is the case, so I’ll plug my own https://www.endbasic.dev/ because it’s very fitting in this context :) I’ve been building it precisely as a way to understand everything that’s going on (although it’s still far from fulfilling that promise).

Also, buried in the article is a reference to the https://10print.org/ book. I recently came across it at HPB and it has been a pretty entertaining read. Couldn’t believe there was so much to write about such a simple little program!