magic-trace
oteps
| magic-trace | oteps | |
|---|---|---|
| 29 | 4 | |
| 4,465 | 320 | |
| 1.1% | 2.2% | |
| 6.8 | 4.8 | |
| 24 days ago | 10 days ago | |
| OCaml | Makefile | |
| MIT License | Apache License 2.0 |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
magic-trace
-
When Optimising Code, Measure
I really like magic-trace [0].
https://github.com/janestreet/magic-trace
Not that the exact tracing relies on Intel PT - support for AMD was added recently but uses perf so suffers from the same sampling/skew issues, but is still very useful.
-
Grafana Phlare, open source database for continuous profiling at scale
Would love to see this integrate with magic trace [1]. I'll need to look at the code for the flamegraph plugin, because handling nanosecond timestamps in flamegraphs seems to break most tools due to float precision.
(1) https://github.com/janestreet/magic-trace
-
How to break into Hudson River Trading?
As for inner workings, have you looked into magic trace? I want to play around with it but last I checked, it doesn't work on macs.
-
Brendan Intel.com
I really hope he can work with cloud vendors and Intel to make Processor Trace a more popular and easier to use capability.
It's unfortunate how https://github.com/janestreet/magic-trace and PMUs in general can't be used by lots of people using cloud VMs.
- GitHub - janestreet/magic-trace: magic-trace collects and displays high-resolution traces of what a process is doing
-
Hacker News top posts: Apr 23, 2022
Magic-trace – High-resolution traces of what a process is doing\ (133 comments)
- Magic-trace – High-resolution traces of what a process is doing
- Magic-trace - 高分辨率跟踪一个进程正在做什么 (Magic-trace – High-resolution traces of what a process is doing)
oteps
-
OpenTelemetry in 2023
Oh nice, thank you (and also solumos) for the links! It looks like oteps/pull/171 (merged June 2023) expanded and superseded the opentelemetry-proto/pull/346 PR (closed Jul 2022) [0]. The former resulted in merging OpenTelemetry Enhancement Proposal 156 [1], with some interesting results especially for 'Phase 2' where they implemented columnar storage end-to-end (see the Validation section [2]):
* For univariate time series, OTel Arrow is 2 to 2.5 better in terms of bandwidth reduction ... and the end-to-end speed is 3.1 to 11.2 times faster
* For multivariate time series, OTel Arrow is 3 to 7 times better in terms of bandwidth reduction ... Phase 2 has [not yet] been .. estimated but similar results are expected.
* For logs, OTel Arrow is 1.6 to 2 times better in terms of bandwidth reduction ... and the end-to-end speed is 2.3 to 4.86 times faster
* For traces, OTel Arrow is 1.7 to 2.8 times better in terms of bandwidth reduction ... and the end-to-end speed is 3.37 to 6.16 times faster
[0]: https://github.com/open-telemetry/opentelemetry-proto/pull/3...
[1]: https://github.com/open-telemetry/oteps/blob/main/text/0156-...
[2]: https://github.com/open-telemetry/oteps/blob/main/text/0156-...
-
Grafana Phlare, open source database for continuous profiling at scale
https://github.com/open-telemetry/oteps/issues/139
It takes a lot of time and effort to bake a cross-vendor cross-language standard.
-
Faster Protocol Buffers
This. The statelessness of the OTLP is by design. I did consider stateful designs with e.g. shared state dictionary compression but eventually chose not to, so that the intermediaries can remain stateless.
An extension to OTLP that uses shared state (and columnar encoding) to achieve more compact representation and is suitable for the last network leg in the data delivery path has been proposed and may become a reality in the future: https://github.com/open-telemetry/oteps/pull/171
What are some alternatives?
viztracer - VizTracer is a low-overhead logging/debugging/profiling tool that can trace and visualize your python code execution.
zipkin-api - Zipkin's language independent model and HTTP Api Definitions
perspective - A data visualization and analytics component, especially well-suited for large and/or streaming datasets.
b3-propagation - Repository that describes and sometimes implements B3 propagation
perfetto - Frontend for magic-trace; forks https://ui.perfetto.dev
odigos - Distributed tracing without code changes. 🚀 Instantly monitor any application using OpenTelemetry and eBPF
linux - Linux kernel source tree
exp-lazyproto - Experimental fast implementation of Protobufs in Go
scalene - Scalene: a high-performance, high-precision CPU, GPU, and memory profiler for Python with AI-powered optimization proposals
community - OpenTelemetry community content
memray - Memray is a memory profiler for Python
terraform-aws-jaeger - Terraform module for Jeager