vswhere VS numexpr

But often maintainers also upload Releases with builds of their software, e.g. like here: https://github.com/microsoft/vswhere/releases

Finding where & how to use an installed VS instance (or selecting one) in automated tooling is solved by the criminally unknown, MIT licensed, MS supported, redistributable, vswhere tool: https://github.com/microsoft/vswhere

/ // This file was about 400 lines before we started adding these comments. // You might think that's way too much code to do something as simple // as finding a few library and executable paths. I agree. However, // Microsoft's own solution to this problem, called "vswhere", is a // mere EIGHT THOUSAND LINE PROGRAM, spread across 70 files, // that they posted to github unironically. // // I am not making this up: https://github.com/Microsoft/vswhere

You can just slap numexpr on top of it to compile this line on the fly.

https://github.com/pydata/numexpr

Are you doing any costly chained NumPy operations in your preprocessing? E.g. max(abs(large_ary)), this produces multiple copies of your data, https://github.com/pydata/numexpr can greatly reduce time spent with such operations

What is not entirely obvious here is that under the hood you can install a nice library called numexpr (docs, src) that exists to make calculations with large NumPy (and pandas) objects potentially much faster. When you use query or eval, this expression is passed into numexpr and optimized using its bag of tricks. Expected performance improvement can be between .95x and up to 20x, with average performance around 3-4x for typical use cases. You can read details in the docs, but essentially numexpr takes vectorized operations and makes them work in chunks that optimize for cache and CPU branch prediction. If your arrays are really large, your cache will not be hit as often. If you break your large arrays into very small pieces, your CPU won’t be as efficient.