opentelemetry-examples VS profiler

We focused on the observability ecosystem and took the time to interact with our friends from Lightstep, New Relic, Honeycomb, Dynatrace, Instana, and many more. With that in mind, keep an eye out for more integrations coming to Tracetest!

Lightstep bills itself as a platform for the reliability of cloud-native applications. The people behind Lightstep co-founded OpenTelemetry and OpenTracing, which gives them a unique perspective on the use cases of distributed tracing and the value of having a vendor-neutral tracing data format.

In the last 5 to 10 years, new Observability vendors have entered the market, including Honeycomb, Instana, Lightstep and Datadog. Similarly, traditional APM vendors such as Dynatrace, AppDynamics, and New Relic, as well as SIEM (and log management) vendors such as Splunk and Sumo Logic, have joined them in the Observability space too. Finally you also have major cloud providers such as AWS with their own observability solution. Each of them is attempting to address the observability issues that modern architecture presents by using Logs, Metrics, Traces, and Events.

I spent Day 2 at the Colony Club to attend OTel Unplugged. This event was sponsored by Lightstep, Honeycomb, New Relic, Splunk, Dynatrace, Crowdstrike, and NGINX. I came into the event not knowing what to expect. I can sometimes clamp up when I’m around folks that I don’t know, but because I was helping with the event check-in, I got to say hello to a number of the attendees, which helped break the ice. And it turns out that there were a lot of names that I recognized from my work in the OTel community, and it was nice to connect in person with folks whom I’d only previously met through Slack or Zoom.

The example in this tutorial can be found in the lightstep/opentelemetry-examples repo. We will be working with three main files:

Note how we don’t have to set a LS_ACCESS_TOKEN, since that’s already configured in the Collector’s config.yml file. Just make sure that you have a running OTel Collector instance!

Note: If you’re looking for full code listings, don’t panic! You see them in the Lightstep OTel examples repository.

Probably your best bet is to use the Firefox Profiler once it starts hogging the CPU to try to see what it's doing. I'd be happy to take a look at the result although I don't have a lot of experience at diagnosing performance problems; hopefully the profile will make it obvious what's going on.

I tried using the https://profiler.firefox.com/ and running with:

You can try to reproduce while taking a profile using the Firefox profiler, then share it in the #perf:mozilla.org room on Mozilla's Matrix instance.

Run a profiling session for ~30 sec when you notice something is running slower than you expect, see https://profiler.firefox.com/. Share results here if you want help interpreting them.

Next time, send a SIGABRT to the main process (kill -6 $(pidof firefox)) and use the resulting backtrace (you will find a link to the crash report in about:crashes) to file a bug. A performance profile could be useful, but you'll need to know first what to capture. This seems like a graphics issue, so try the Graphics preset. Share a link to your report.

https://profiler.firefox.com will be useful if you can capture and share it to developers.

If you can reproduce it in safe mode and a clean profile, capture a performance profile using the clean profile while trying to load a problematic webpage, don't remove any information from it, and share it here (although it's typically not easy for users to analyse them), or file a bug with it attached.

Next time, capture a performance profile using the Graphics preset, and kill the browser by entering kill -6 $(pidof firefox) in a shell. This will create a bug report (unless debian disables the crash reporter) that would indicate where Firefox is stuck at. You might need to use an official build to get useful results.