Opentelemetry-go Alternatives

opentelemetry-go reviews and mentions

• Tracing: Structured Logging, but better in every way
  9 projects | news.ycombinator.com | 20 Sep 2023

  Indeed, the three legs (metrics, logs, traces) of OpenTelemetry's telescope. https://opentelemetry.io

• Show HN: HyperDX – open-source dev-friendly Datadog alternative
  12 projects | news.ycombinator.com | 18 Sep 2023

  PS- I just skimmed https://opentelemetry.io, which your readme.md links to.

  Good stuff. Much industry progress since I was last in the arena.

  Their site has words about manual and automatic instrumentation. I'd have to dig a bit to see what they mean.

  --

  So. Remembering a bit more... Will try to keep this brief; you're a busy person.

  > tend to log useless information or fail to tag them in ways that are actually searchable

  #1 - I don't know know to manage lifecycle of meta. Who needs what? When is it safe to remove stuff?

  We logged a lot of URLs. So many URL params. And when that wasn't crazy enough, over flow into HTTP headers. Plus partially duplicate, incorrectly, info in the payloads, a la SOAP. ("A person with two watches has no idea what time it is.")

  When individual teams were uncertain, they'd just forward everything they received (copypasta), and add their own stuff.

  Just add correlation IDs, right?

  Ah, but there's "legacy". And unsupported protocols, like Redis and JDBC. And brain dead 3rd party services, with their own brain dead CSRs and engrs.

  This is really dumb, and just propagates badness, but a few times, in a pinch, I've created Q&D "logging proxy". Just to get some visibility. So dumb.

  Why stop there? Just have "the fabric" record stuff. Repurpose Wireguard into an Omni Logger. (Like the NSA does. Probably.) That'd eliminate most I/O trace style logging, right?

  Image all these "webservices" and serverless apps without any need for instrumentation. Just have old school app level logging.

  #2 - So much text processing.

  An egregious example is logging HTTP headers. Serialize them as JSON and send that payload to a logging service. Which then rehydrate and store it some where.

  My radical idea, which exactly no one has bought into, is to just pipe HTTP (Requests and Responses) as-is to log files. Then rotate, groom, archive, forward, inject, compress, whatever as desired.

  That's what I did on the system I mentioned. All I/O was just streamed to files. And in the case of the HL7 (medical records stuff), it was super easy to extract the good bits, use that for Lucene's metadata, and store the whole message as the Lucene document.

  I know such a radical idea is out of scope for your work. Just something fun to think about.

  #3

  > if none of your logs let you link service performance to customer X

  Yup. Just keep adding servers. Kick the can down the road.

  One team I helped had stuff randomly peg P95. And then sometimes a seemingly unrelated server would tip over. Between timeouts, retries, and load balancers, it really seemed like the ankle bone was connected to the shoulder bone. It just made no sense.

  Fortunately, I had some prior experience. Being new to nodejs, maybe 5 years ago, I was shocked to learn there was no notion of back pressure. It was a challenging concept to explain to those teammates. But the omission of backpressure, and a hunch, was a good place for me start. (I'm no Dan Luu or Bryan Cantrill.)

  I'd like to think that proper end-to-end logging, and the ability to find signal in the noise, diagnosis would have been more mundane.

• Distributed Traces for Testing with Tekton Pipelines and Tracetest
  5 projects | dev.to | 8 Sep 2023

  Tracetest, an open-source testing tool that uses OpenTelemetry traces for testing, offers a sophisticated test harness for distributed cloud-native apps. It empowers users to test their apps by harnessing data from distributed traces produced by OpenTelemetry. This enables creating test specs and assertions that validate whether an application aligns with the intended behavior, as defined by pre-established test parameters.

• OpenTelemetry in 2023
  36 projects | news.ycombinator.com | 28 Aug 2023

  https://opentelemetry.io

  > OpenTelemetry is a collection of APIs, SDKs, and tools. Use it to instrument, generate, collect, and export telemetry data (metrics, logs, and traces) to help you analyze your software’s performance and behavior.

  You can absolutely categorize telemetry into these high-level categories, true. But the specifics on how that data is captured, exported, collected, queried, etc. is necessarily unique to each programming language, backend system, organization, etc.

  That's because telemetry data is always larger than the original data it represents: a production request will be of some well-defined size, but the metadata about that request is potentially infinite. Consequently, the main design constraint for telemetry systems is always efficiency.

  Efficiency requires specialization, which is in direct tension with features that generalize over backends and tools, e.g.

  > Traces, Metrics, Logs -- Create and collect telemetry data from your services and software, then forward them to a variety of analysis tools.

  and features that generalize over languages, e.g.

  > Drop-In Instrumentation -- OpenTelemetry integrates with popular libraries and frameworks such as Spring, ASP.NET Core, Express, Quarkus, and more! Installation and integration can be as simple as a few lines of code.

  I think OTel treats these goals -- which are very valuable to end users!! -- as inviolable core requirements, and then does whatever is necessary to implement them, even if the resulting code is unsound, or inefficient, or incoherent.

  36 projects | news.ycombinator.com | 28 Aug 2023

  Try implementing an OTEL Tracer. This interface is insane, and should really just be a struct.

  https://github.com/open-telemetry/opentelemetry-go/blob/trac...

• Launch HN: Highlight.io (YC W23) – Open-source, full stack web app monitoring
  15 projects | news.ycombinator.com | 18 Jul 2023

  A bit of background: Vadim and I have worked at quite a few startups at this point, and a recurring challenge we’ve faced was tracing usability issues on the frontend to downstream errors and logs on the server. Understanding the real reason behind customer issues was always a chaotic juggling of multiple tools. With the rise of "frontend-forward" frameworks such as NextJS, which blur the boundary between the client and server, the complexity of tracing these issues is only growing.

  This is where Highlight.io comes in: our product bridges the gap between client and server to give you a holistic view of your entire application.

  At its core, Highlight.io has three main “products”: Session Replay, Error Monitoring, and Logging. The novelty here is not in each product but in how they are connected. For example, in Highlight.io it’s very easy to click from a given error to the associated user session where it is thrown [4], and from a given error, you can easily inspect all of the logs that fired leading up to it. Ensuring that all of our products work together seamlessly with little to no effort is a core principle of our product strategy. If you’re using a common framework [5], for example, we’ll automatically link your frontend sessions with backend errors and logs. No agents, configuring facets, or anything else, It just works.

  We depend on several open source projects that help us move quickly. OpenTelemetry (OTEL) [6] is one of them, which helps us with maintainability, i.e. for every language that we support, we only maintain a thin wrapper around its respective OTEL SDK. OTEL is also a great way to enable the community to contribute, and we’re already seeing traction in this space (ie. an open source contributor built a wrapper for a Java SDK [7]).

  rrweb [8] is another project we leverage heavily for our session replay product. It drives our ability to record and replay the DOM to visualize user flows in the frontend. We’ve had the privilege to work closely with the rrweb team to ship improvements, and we’re now actively sponsoring the project [9].

  ClickHouse [10] has recently become a loved database on our team, as we historically used Opensearch for search-heavy workloads and started to hit growing pains with ingest throughput. We recently rolled it out for our logging product [3] and plan to replace our sessions and errors (and upcoming tracing work) with the database as well.

  From a business perspective, Highlight.io is open source under the Apache 2.0 license, and we make money with our hosted product [11]. For the hosted product, you can set billing caps for each offering and we don’t charge for seats. At this point, we have 100+ companies paying for our product (some of which are large enterprises), and thousands of sole developers use Highlight.io every week.

  On our roadmap [12] for the future includes metrics, tracing, release management and more. We also are launching several updates this week on our launch week page [13].

  Overall, we’re excited to be sharing Highlight.io with the world, and Vadim and I are particularly excited to get some feedback from the HN community. Please give us a test-drive at https://app.highlight.io and let us know what you think. We would love to learn about what you wish you had in an observability product as well as any other experiences and ideas in this space. We look forward to hearing from you!

  1: https://github.com/highlight/highlight/blob/main/LICENSE

  2: https://news.ycombinator.com/item?id=34897645

  3: https://news.ycombinator.com/item?id=35643255

  4: https://youtu.be/EvGsmbt0F7s?t=65

  5: https://highlight.io/frameworks

  6: https://opentelemetry.io/

  7: https://github.com/highlight/highlight/pull/4812

  8: https://github.com/rrweb-io/rrweb

  9: https://github.com/rrweb-io/rrweb#gold-sponsors-

  10: https://clickhouse.com/blog/overview-of-highlightio

  11: https://highlight.io/pricing

  12: https://www.highlight.io/docs/general/roadmap

  13: https://highlight.io/launch-week-2

• Trace-based Testing AWS Lambda with Tracetest, ECS Fargate, and Terraform
  8 projects | dev.to | 7 Jul 2023

  Today, standard protocols like OpenTelemetry exist to help teams implement observability as part of the development process and provide a clear understanding of what their serverless application is doing.

• Tracetest Analyzer: Identify patterns and issues with code instrumentation
  3 projects | dev.to | 7 Jul 2023

  The Tracetest Analyzer is an evaluation framework composed of plugins. It analyzes OpenTelemetry traces to help teams improve instrumentation data, find potential problems and provide tips to fix problems with code instrumentation.

• Hands-on OpenTelemetry: Troubleshoot issues with your instrumented apps
  5 projects | dev.to | 27 Jun 2023

  OpenTelemetry is an open source project (opentelemetry.io) and unified standard for service instrumentation that provides an application programming interface (API), libraries, integrations, and a software development kit (SDK) for programming languages including Java, Go, and Python.

• Is anyone frustrated with anything about Prometheus?
  5 projects | /r/PrometheusMonitoring | 18 Jun 2023

  We're fully aware that a large observability ecosystem exists outside of Google and that offering a brand new foundational component of the stack is a very ambitious goal, but OTOH there are use cases that simply cannot be addressed by Prometheus (pretty much like they couldn't be addressed by Borgmon). Besides, we plan to integrate with existing leading technologies such as OpenTelemetry and ease hypothetical migrations e.g. by writing a scraper for Prometheus clients that actually writes data to our database.

• A note from our sponsor - Mergify
  blog.mergify.com | 28 Sep 2023

  Solutions like Dependabot or Renovate update but don't merge dependencies. You need to do it manually while it could be fully automated! Add a Merge Queue to your workflow and stop caring about PR management & merging. Try Mergify for free. Learn more →