pytest-codspeed VS FlameGraph

CodSpeed | Founding AI Engineer | On-site (Paris) / Remote (Europe) | Full-time | https://codspeed.io

We're building software performance optimization tools to optimize and measure code performance before it is deployed to production. We avoid regressions that impact UX and help developers solve their performance issues faster. We're already live and trusted by top-tier open-source project teams such as Pydantic, Ruff, and Prisma.

We’re at an exciting early stage and looking for talented engineers who share our passion for helping to enhance the performance of software used by billions, improving the software development lifecycle, and building tools we love to use ourselves.

Apply at https://codspeed.notion.site/Founding-AI-Engineer-cd1bf4fd73...

pytest-codspeed, plugin for pytest

https://github.com/brendangregg/FlameGraph

You record performance data with `perf`, then use the scripts there to turn it into a SVG.

If you'd like to know more about the flame graphs in general you can you the collection of resources of Brendan Gregg Flame Graph who contributed a lot in this area.

You may find yourself in need of flamegraphs for software in this repository. You can use flamegraph-rs or the original flamegraph.pl. Your choice!

The timeline view in the upper screenshot is fairly straightforward: the region of 28-53s is selected, other parts of the timeline are grayed out, and the statistics for the selected time region down below show 94.4% of "Total CPU" being in id_sort_by_name.

The lower screenshot is a flame graph. If you haven't encountered one of those before, it's totally reasonable to not know how to read it. But it's a standard and common way to present this sort of data. See https://www.brendangregg.com/flamegraphs.html for more information.

Looking at the result for aligned writes, it takes 6.3s in total to complete. Of this, 0.3s is used in user land with 6s used by the system in kernel space(syscalls). In order to make sense of the results, I profiled and collected the stacks traces for analysis. The call stacks can be visualized in the flamegraphs as shown below

If you want to "go low" in the operating system, you could write a Linux kernel module:

https://sysprog21.github.io/lkmpg/

https://blog.sourcerer.io/writing-a-simple-linux-kernel-modu...

If you want to "go low" in the way hardware works, you could try and write an interrupt handler on an embedded device.

If you want to "go low" in how optimizations work in application development, you could try and implement microbenchmarks and look at flamegraphs.

https://github.com/brendangregg/FlameGraph

https://bheisler.github.io/criterion.rs/book/index.html

We'll use async-profiler and flame graphs for profiling. To simplify the process, we'll run the code using JBang.

And flame graphs excel and this kind of thing

https://www.brendangregg.com/flamegraphs.html

which can output in a format understood by Brendan Gregg's flame frames (https://www.brendangregg.com/flamegraphs.html)

But that's not quite the kind of tracing you're talking about. We also built a printf-style interface to our recording files, which seems closer:

Strategies included discontinuing or resizing underutilized services, transitioning to more cost-effective solutions, reducing the current resources to the amount of resources that we need for our application, and conducting detailed analyses of computing resource utilization through tools like flamegraphs. This detailed scrutiny helped identify and rectify significant cost-driving areas, such as garbage collection and application configurations.