coveragepy VS scalene

Coverage.py is the go-to tool for measuring code coverage of Python programs. Once installed, you can use it with either unittest or pytest.

Code Coverage Analysis assess the code portions tested by the current test suites without altering the code.

The PLASMA lab @ UMass Amherst (home of the Scalene profiler) has released a new version of Slipcover, a super fast code coverage tool for Python. It is by far the fastest code coverage tool: in our tests, its average slowdown is just 5% (compare to the widely used coverage.py, average slowdown 218%!). The latest release performs both line and branch coverage with virtually no overhead. Use it to dramatically speed up your tests and continuous integration!

Tests ensure the tested behavior is maintained. It's up to the developers to write tests with sufficient coverage. Determining which lines of code on your project are covered by tests is easily quantifiable using tooling. E.g. https://coverage.readthedocs.io/

Taking the idea further, if you build robust tests for your API, you could use a dynamic code analyzer like coverage or figleaf to identify and delete unused functions.

coverage

FWIW, ternary expressions aren't properly detected by coverage: https://github.com/nedbat/coveragepy/issues/509

This has been a feature of the Scalene Python profiler (https://github.com/plasma-umass/scalene) for some time (at this point, about 1.5 years) - bring your own API key for OpenAI / Azure / Bedrock, also works with Ollama. Optimizing Python code to use NumPy or other similar native libraries can easily yield multiple order of magnitude improvements in real-world settings. We tried it on several of the success stories of Scalene (before the integration with LLMs); see https://github.com/plasma-umass/scalene/issues/58 - and found that it often automatically yielded the same or better optimizations - see https://github.com/plasma-umass/scalene/issues/554. (Full disclosure: I am one of the principal designers of Scalene.)

Using Python for glue code and compiled native code (whether C or C++ or Rust or whatever) is a classic strategy.

Just profile your code with something like Scalene: https://github.com/plasma-umass/scalene

Alternatively, you can just write it in Julia.

I really want to know what the optimizations might have looked like had they used a profiler like scalene [0] to find where the unnecessary copying was happening.

[0] https://github.com/plasma-umass/scalene

Use this my fren: https://github.com/plasma-umass/scalene

You should take a look at Scalene - it's even better.

https://github.com/plasma-umass/scalene

I like seeing another Python profiler. The one I've been playing with is Scalene (GitHub). It does some fun things related to letting you see how much things are moving across the system Python memory boundary.