self-hosted VS opentelemetry-specification

I was responding to the One of the Sentry inconvenience is self-hosting: it relies on so many services it can be very complicated to maintain part, and also reminding readers that if they, too, hate companies that rug-pull their open source licenses, there is a band-aid for both parts

Compare https://github.com/getsentry/self-hosted/blob/9.1.2/docker-c... with https://github.com/getsentry/self-hosted/blob/24.4.2/docker-... for what life used to be like for running Sentry on-prem. It was awesome

It would take a ton of work to dig up the actual memory and CPU requirements of each one, but rest assured they're not zero, so every one of those services eats ram and requires TLC when, not if, they shit themselves. So, more parts == more headaches with all other things being equal

Then, I deeply appreciate that there are a whole spectrum of reactions to the various licensing schemes in use nowadays, and a bunch of folks don't care. I care, though, because I have gotten immense value from open source projects, and have contributed changes back to quite a few. It has been my life experience that any of those "source available" licenses usually are very hostile toward making local builds and if I can't build it to match how prod goes, then I can't test my fixes in my environment and then I can't contribute the PR with any faith

This is the point where I will point out that you can self-host Sentry free of charge :) https://develop.sentry.dev/self-hosted/

Sentry can be self hosted: https://develop.sentry.dev/self-hosted/

The people we're concerned about are not the hundreds of thousands of Sentry users, including those that self-host.

We're concerned about people who have taken the software for the purposes of competing directly against us, that hinders our ability to monetize the work. Monetizing the work helps us continue improving the software and distribute it for free use, benefitting those aforementioned real users (e.g. https://github.com/getsentry/self-hosted).

> I mean no slander or disrespect to anyone involved, but there was a DataDog alternative posted sometime in the last few weeks that had a docker-compose with like 15 containers in it.

Reminds me of Sentry: https://develop.sentry.dev/self-hosted/

This is their example docker-compose for self-hosting: https://github.com/getsentry/self-hosted/blob/master/docker-...

It has:

  - exim4 (smtp)

> What should people use?

I recall Apache Skywalking being pretty good, especially for smaller/medium scale projects: https://skywalking.apache.org/

The architecture is simple, the performance is adequate, it doesn't make you spend days configuring it and it even supports various different data stores: https://skywalking.apache.org/docs/main/v9.0.0/en/setup/back...

The problems with it are that it isn't super popular (although has agents for most popular stacks), the docs could be slightly better and I recall them also working on a new UI so there is a little bit of churn: https://skywalking.apache.org/downloads/

Still better versus some of the other options when you need something that just works instead of spending a lot of time configuring something (even when that something might be superior in regards to the features): https://github.com/getsentry/self-hosted/blob/master/docker-...

Sentry is just the first thing that comes to mind (OpenTelemetry also isn't simpler due to how much it tries to do), but compare its complexity to Skywalking: https://github.com/apache/skywalking/blob/master/docker/dock...

I wish there was more self-hosted software like that out there, enough to address certain concerns in a simple way on day 1 and leave branching out to more complex options like OpenTelemetry once you have a separate team for that and the cash is rolling in.

Sounds interesting!

Will you compare with qryn? Self-hosted sentry?

qryn.metrico.in/

https://develop.sentry.dev/self-hosted/

I haven't done it in years, but technically sentry is able to be self hosted https://github.com/getsentry/self-hosted

It means that the OpenTelemetry project provides not only a specification to define the contract between the applications, collectors, and telemetry databases, but also a set of APIs, SDKs, and tools like instrumentation libraries (for different languages), collectors, operators, etc. OpenTelemetry is open-source and vendor-agnostic, so the project is not tied to any specific vendor or cloud provider.

Sure, happy to provide more specifics!

Our main issue was the lack of a synchronous gauge. The officially supported asynchronous API of registering a callback function to report a gauge metric is very different from how we were doing things before, and would have required lots of refactoring of our code. Instead, we wrote a wrapper that exposes a synchronous-like API: https://gist.github.com/yolken-airplane/027867b753840f7d15d6....

It seems like this is a common feature request across many of the SDKs, and it's in the process of being fixed in some of them (https://github.com/open-telemetry/opentelemetry-specificatio...)? I'm not sure what the plans are for the golang SDK specifically.

Another, more minor issue, is the lack of support for "constant" attributes that are applied to all metrics. We use these to identify the app, among other use cases, so we added wrappers around the various "Add", "Record", "Observe", etc. calls that automatically add these. (It's totally possible that this is supported and I missed it, in which case please let me know!).

Overall, the SDK was generally well-written and well-documented, we just needed some extra work to make the interfaces more similar to the ones were were using before.

OpenTelemetry is an open-source collection of tools, APIs, and SDKs that aims to standardize the way we generate and collect telemetry data. It follows a specification-driven development. The OpenTelemetry specification  has design and implementation guidelines for how the instrumentation libraries should be implemented. In addition, it provides client libraries in all the major programming languages that follow the specification.

Two problems with OpenTelemetry:

1. It doesn't know what the hell it is. Is it a semantic standard? Is a protocol? It is a facade? What layer of abstraction does it provide? Answer: All of the above! All the things! All the layers!

2. No one from OpenTelemetry has actually tried instrumenting a library. And if they have, they haven't the first suggestion on how instrumenters should actually use metrics, traces, and logs. Do you write to all three? To one? I asked this question two years ago, not a single response. [1]

[1] https://github.com/open-telemetry/opentelemetry-specificatio...

OpenTelemetry Specification GitHub

One update to this: we proposed replacing the count suffix in OpenTelemetry with total to match Prometheus/OpenMetrics. That discussion resulted in the count suffix being removed from the OpenTelemetry semantic conventions. We'll soon update our metric from being called function.calls.count to just function.calls and the generated Prometheus queries will refer to function_calls_total. That resolves one of the main conflicts between the two specs.

This is your best bet if you want to track status updates: https://github.com/open-telemetry/opentelemetry-specification/issues/2911

OpenTelemetry is a standard for implementing telemetry in your applications. It provides a specification, containing the requirements that all implementations should follow as well as some implementations for major languages, including an API and a SDK to interact with it.

At this point, we are about to do the real work where we have to configure OpenTelemetry and export telemetry data to New Relic. Exporting this kind of data relies on a specific protocol; the OpenTelemetry Protocol or OTLP.

OpenTelemetry provides instrumentation libraries for your application. The development of these libraries is guided by the OpenTelemetry specification. The OpenTelemetry specification describes the cross-language requirements and design expectations for all OpenTelemetry implementations in various programming languages.