Codespeed Alternatives

codespeed reviews and mentions

• Adding runtime benchmarks to the Rust compiler benchmark suite
  3 projects | news.ycombinator.com | 30 Sep 2023

  I've looked into this before and there are very few tools for this. The only vaguely generic one I've found is Codespeed: https://github.com/tobami/codespeed

  However it's not very good. Seems like most people just write their own custom performance monitoring tooling.

  As for how you actually run it, you can get fairly low noise runtimes by running on a dedicated machine on Linux. You have to do some tricks like pinning your program to dedicated CPU cores and making sure nothing else can run on them. You can get under 1% variance that way, but in general I found you can't really get low enough variance on wall time to be useful in most cases, so instruction count is a better metric.

  I think you could do better than instruction count though but it would be a research project - take all the low noise performance metrics you can measure (instruction count, branch misses etc), measure a load of wall times for different programs and different systems (core count, RAM size etc.). Feed it into some kind of ML system and that should give you a decent model to get a low noise wall time estimate.

  Good tips here:

  https://llvm.org/docs/Benchmarking.html

  https://easyperf.net/blog/2019/08/02/Perf-measurement-enviro...

• how to benchmark a programming language
  5 projects | /r/compsci | 22 Jan 2023
• Is there a standard (similar to JUnit) for tracking project performance between releases?
  1 project | /r/AskProgramming | 9 Feb 2022

  The only thing I've found after modest research is codespeed as used by https://speed.python.org

• A note from our sponsor - SaaSHub
  www.saashub.com | 9 May 2024

  SaaSHub helps you find the best software and product alternatives Learn more →