openobserve VS opentelemetry-go

Lot of interesting OSS observability products coming out in recent years. One of the more impressive(and curious for many reasons) IMHO is OpenObserve: https://github.com/openobserve/openobserve .

As opposed to just a stack, they are implementing just about the whole backend shebang from scratch.

in case it matters to others, https://github.com/openobserve/openobserve/tree/v0.7.0 is the last Apache2 licensed copy before they went AGPL with 0.7.1

https://github.com/openobserve/openobserve/blob/v0.7.0/.env.... is some "onoz" for me, but just recently someone submitted https://github.com/aenix-io/etcd-operator to the CNCF sandbox so maybe things have gotten better around keeping that PoS alive

eCharts is awesome. We moved from plotly after using it for several months to echarts at https://github.com/openobserve/openobserve and are super happy.

Wouldn't make more sense to have the same observability stack on production and development? For instance, open-observe is also a single binary that provides UI for logs, metrics and traces, although every log producer would have to be properly configured and routing to it.

Another idea: maybe chrome dev-tools could be repurposed to display server logs instead of client logs, somehow [2].

--

1: https://github.com/openobserve/openobserve

2: https://chromedevtools.github.io/devtools-protocol/

It doesn't read from files unfortunately, but https://openobserve.ai/ is very easy to set up locally (single binary) and send otel logs/metrics/traces to.

Here's how I run it locally for my little shovel project - https://github.com/bbkane/shovel#run-the-webapp-locally-with... .

Also linked from that README is an Ansible playbook to start OpenObserve as a systems service on a Linux VM.

Alternatively, see the shovel codebase I linked above for a "stdout" TracerProvider. You could do something like that to save to a file, and then use a tool to prettify the JSON. I have a small script to format json logs at https://github.com/bbkane/dotfiles/blob/2df9af5a9bbb40f2e101...

Implement a monitoring stack, or openobserve for an all-in-one package.

I would recommend you take a look at OpenObserve (https://github.com/openobserve/openobserve ). It's free and open source and can do all you asked and more with far lower resource utilization. It's the easiest to run of any log system that you can find. Can capture windows and linux logs. Also compresses them heavily (30-60x, YMMV). 100 GB ingested logs can be 3 GB stored.

I have used https://github.com/openobserve/openobserve in several hobby projects and liked it. It's an all-in-one solution. It's likely less featureful than many others but a single binary and everything in one place pulled me in and worked for me so far.

Not affiliated, I just like the tool.

A good one. A lot is being built on top of clickhouse. I can count at least 3 if not more (hyperdx, signoz and highlight) built on top of clickhouse now.

We at OpenObserve are solving the same problem but a bit differently. A much simpler solution that anyone can run using a single binary on their own laptop or in a cluster of hundreds of nodes backed by s3. Covers logs, metrics, traces, Session replay, RUM and error tracking are being released by end of the month) - https://github.com/openobserve/openobserve

Our first approach was to implement a separate SDK for each independent technology stack. We decided to use OpenTelemetry which is widely adopted and covers most of our needs.

Distributed system administrators need mechanisms and tools for monitoring individual nodes in order to analyze the system and promptly detect anomalies. Developers also need effective mechanisms for analyzing, diagnosing issues, and identifying bugs in protocol implementations. Logging, tracing, and collecting metrics are common observability techniques to allow monitoring and obtaining diagnostic information from the system; most of the explored code bases use these techniques. OpenTelemetry and Prometheus are popular open-source monitoring solutions, which are used in many of the explored code bases.

OpenTelemetry

When choosing distributed tracing tools, considerations include your technology stack, business requirements, and monitoring complexity. Zipkin, SkyWalking, and OpenTelemetry are popular distributed tracing solutions, each with its unique features.

You can follow this process with any large token AI system like Claude by identifying tracing data relevant to the code you are working on, using it as context to prompt OpenAI or other LLMs. Generally, you’d generate tracing data by implementing OpenTelemetry (aka OTEL) libraries into your application, adding spans to your functions with Jaeger, or using commercial SaaS tools like Honeycomb and Datadog.

While many programming languages provide robust support for Open Telemetry, this instance focuses on Golang. It's important to note that, in the current context, the logs SDK for Golang is not implemented. For future reference consult the list of supported languages and explore the Open Telemetry repositories. Always prioritize the main repository and its contrib repository, housing extensions and instrumentation libraries crucial to the Open Telemetry framework. Stay updated with the latest developments to ensure seamless integration and enhanced functionality.

OneUptime (https://github.com/oneuptime/oneuptime) is the open-source alternative to DataDog. It's 100% free and you can self-host it on your VM / server / cloud or you can use SaaS at https://oneuptime.com

NEW UPDATES (since we last posted to HN): We now support OpenTelemetry (https://opentelemetry.io/) natively which will help you to monitor, observe and debug any app, service, database or stack.

OpenTelemetry is the fastest growing Cloud Native Computing Foundation (CNCF) project. It standardizes the instrumentation and collection of traces, metrics, and logs from applications, and is supported by all the major observability projects, languages, and tools. One standard to rule them all!

OpenTelemetry is a collection of tools, APIs, and SDKs. Use it to instrument, generate, collect, and export telemetry data (metrics, logs, and traces) to help you analyze your software’s performance and behavior. https://opentelemetry.io/

Tracetest uses your existing OpenTelemetry traces to power trace-based testing with assertions against your trace data at every point of the request transaction. You only need to point Tracetest to your existing trace data source, or send traces to Tracetest directly!