zlib-ng VS NumPy

Please note that allowing for 2% bigger resulting file could mean huge speedup in these circumstances even with the same compression routines, seeing these benchmarks of zlib and zlib-ng for different compression levels:

https://github.com/zlib-ng/zlib-ng/discussions/871

IMO the fair comparison of the real speed improvement brought by a new program is only between the almost identical resulting compressed sizes.

It is much faster than miniz_oxide and all other safe-Rust implementations, and consistently beats even Zlib. The performance is roughly on par with zlib-ng - sometimes faster, sometimes slower. It is not (yet) as fast as the original libdeflate in C.

Zlib-ng doesn't contain the same code, but it appears that their equivalent inflate() when used with their inflateGetHeader() implementation was affected by a similar problem: https://github.com/zlib-ng/zlib-ng/pull/1328

Also similarly, most client code will be unaffected because `state->head` will be NULL, because they (most client code) won't have used inflateGetHeader() at all.

I wonder if zlib-ng would make a difference, since it has a lot of optimizations for modern hardware.

https://github.com/zlib-ng/zlib-ng/discussions/871

zlib-ng also has adler32 implementations optimized for various architectures: https://github.com/zlib-ng/zlib-ng

Might be interesting to benchmark their implementation too to see how it compares.

zlib-ng: https://github.com/zlib-ng/zlib-ng/blob/develop/functable.c

(2) There are many packages that rely upon zlib and minizip and switching those underlying dependencies is easier said than done. We can't drop zlib completely and switch: "The idea of zlib-ng is not to replace zlib, but to co-exist as a drop-in replacement with a lower threshold for code change." - https://github.com/zlib-ng/zlib-ng

There are already active zlib forks (e.g. https://github.com/zlib-ng/zlib-ng), the problem is with having people move to them. It takes a lot of effort to move mindshare from the original version to a fork, there's some historical examples of it happening, but not a ton.

In NumPy with @, dot() or matmul():

In NumPy with * or multiply(). ` or multiply()` can multiply 0D or more D arrays by element-wise multiplication.

Data science projects often use numpy. However, numpy objects are not JSON-serializable and therefore require conversion to standard python objects in order to be saved:

numpy: A library for scientific computing in Python

Python has become a preferred language for data analysis due to its simplicity and robust library ecosystem. Among these, NumPy stands out with its efficient handling of numerical data. Let’s say you’re working with numbers for large data sets—something Python’s native data structures may find challenging. That’s where NumPy arrays come into play, making numerical computations seamless and speedy.

I was looking at all the big projects like numpy, pytorch, flask, etc.

I wish the NumPy devs would more thoroughly consider adding full fluent API support, e.g. x.sqrt().ceil(). [Issue #24081]