criterion.rs VS flamegraph

The three popular options for benchmarking in Rust are: libtest bench, Criterion, and Iai.

Holy crap. I was just writing a blog to complain about the state of Rust benchmarking and I think this might address most of my points. The biggest one is the ability to have benchmarks collocated within the library like tests which is the biggest annoyance.

It’s also nice to see that it can report multiple counters in parallel. I put up a similar feature[1] for criterion recently but I fear the project isn’t being maintained anymore…

Haven’t looked deeply into divan yet but the other requirements I have for criterion’s power is to run tests with statistical guarantees on the results, terminate quickly when statistical significance is reached (—quick), provide a comparison of the delta from a previous benchmark, and to run async code. Wonder how this stacks up.

[1] https://github.com/bheisler/criterion.rs/pull/722

criterion for benchmarks,

For an extra optimization, you might look into techniques like branchless code, turning array of structs into struct of arrays for better cache utilization/lowering branch mispredictions. There is many talks on YouTube how to actually measure and improve performance of native code. And for rust there is a criterion.rs for benchmarking.

https://github.com/bheisler/criterion.rs is good for tests like that. It will give you much more than a single number and handle things like outliers. This makes identifying noisy tests simpler.

I have tried many ways but I think is not possible. I put it on https://github.com/bheisler/criterion.rs/issues/631.

Criterion is still the gold standard.

Pros for Criterion over the stdlib: https://github.com/bheisler/criterion.rs#features

Downsides of Criterion: https://bheisler.github.io/criterion.rs/book/user_guide/know...

You can install cargo-flamegraph with cargo install flamegraph. There are some underlying requirements to be able to use cargo-flamegraph; you will want to take a look at the repo here to make sure you have the right dependencies.

I tried to diagnose the issue with flamegraph, but unfortunately the flamegraph didn't show anything beyond the next call for some reason

having some tools to identify slowness origins (flamegraph is one... but not sure it's the way to go)

I'm currently implementing a solution for the first part of the day 16. It work but it is really slow... I'd like to : - understand why - having some tools to identify slowness origins (flamegraph is one... but not sure it's the way to go) - eventually have some clue/solution/idea - have general feedback on what in my "coding style" is not appropriate for rust (I come from java/kotlin/ts even if I've already coded a bit in c/c++) : for example I love iterator & sequence but i feel they are not really suited to overuse in rust (mostly because of async & result).

Use a profiler. Flamegraph is a good way to visualise profiler output. This lets you identify which functions are taking up a large amount of time - and hence helps you identify where to focus your optimisation efforts.

You tried trying to see what takes the most time under load via flames? https://github.com/flamegraph-rs/flamegraph

Well, in cases like that the answer is straight forward, use a profiler like https://github.com/flamegraph-rs/flamegraph

I don't see any obvious problems with your algorithm. I've had luck using cargo-flamegraph to identify the slow parts of my code. That's going to show you which parts to focus on improving the performance of!

From the readme of cargo flamegraph: