Benchmarking 20 programming languages on N-queens and matrix multiplication

1. plb2

   1 8 260 2.4 C

   A programming language benchmark

   

   A curious thing about Swift: after https://github.com/attractivechaos/plb2/pull/23, the matrix multiplication example is comparable to C and Rust. However, I don’t see a way to idiomatically optimise the sudoku example, whose main overhead is allocating several arrays each time solve() is called. Apparently, in Swift there is no such thing as static array allocation. That’s very unfortunate.

2. CodeRabbit

   coderabbit.ai featured

   CodeRabbit: AI Code Reviews for Developers. Revolutionize your code reviews with AI. CodeRabbit offers PR summaries, code walkthroughs, 1-click suggestions, and AST-based analysis. Boost productivity and code quality across all major languages with each PR.

   

3. laser

   2 7 281 3.6 Nim

   The HPC toolbox: fused matrix multiplication, convolution, data-parallel strided tensor primitives, OpenMP facilities, SIMD, JIT Assembler, CPU detection, state-of-the-art vectorized BLAS for floats and integers (by mratsim)

   

   ```

   Note: the Theoretical peak limit is hardcoded and used my previous machine i9-9980XE.

   It maybe that your BLAS library is not named libopenblas.so, you can change that here: https://github.com/mratsim/laser/blob/master/benchmarks/thir...

   Implementation is in this folder: https://github.com/mratsim/laser/tree/master/laser/primitive...

   in particular, tiling, cache and register optimization: https://github.com/mratsim/laser/blob/master/laser/primitive...

   AVX512 code generator: https://github.com/mratsim/laser/blob/master/laser/primitive...

   And generic Scalar/SSE/AVX/AVX2/AVX512 microkernel generator (this is Nim macros to generate code at compile-time): https://github.com/mratsim/laser/blob/master/laser/primitive...

   I'll come back later with details on how to use my custom HPC threadpool Weave instead of OpenMP (https://github.com/mratsim/weave/tree/master/benchmarks/matm...)

4. weave

   3 8 550 1.7 Nim

   A state-of-the-art multithreading runtime: message-passing based, fast, scalable, ultra-low overhead (by mratsim)

   

   ```

   Note: the Theoretical peak limit is hardcoded and used my previous machine i9-9980XE.

   It maybe that your BLAS library is not named libopenblas.so, you can change that here: https://github.com/mratsim/laser/blob/master/benchmarks/thir...

   Implementation is in this folder: https://github.com/mratsim/laser/tree/master/laser/primitive...

   in particular, tiling, cache and register optimization: https://github.com/mratsim/laser/blob/master/laser/primitive...

   AVX512 code generator: https://github.com/mratsim/laser/blob/master/laser/primitive...

   And generic Scalar/SSE/AVX/AVX2/AVX512 microkernel generator (this is Nim macros to generate code at compile-time): https://github.com/mratsim/laser/blob/master/laser/primitive...

   I'll come back later with details on how to use my custom HPC threadpool Weave instead of OpenMP (https://github.com/mratsim/weave/tree/master/benchmarks/matm...)

5. array

   4 5 200 3.6 C++

   C++ multidimensional arrays in the spirit of the STL

   

   I should have mentioned somewhere, I disabled threading for OpenBLAS, so it is comparing one thread to one thread. Parallelism would be easy to add, but I tend to want the thread parallelism outside code like this anyways.

   As for the inner loop not being well optimized... the disassembly looks like the same basic thing as OpenBLAS. There's disassembly in the comments of that file to show what code it generates, I'd love to know what you think is lacking! The only difference between the one I linked and this is prefetching and outer loop ordering: https://github.com/dsharlet/array/blob/master/examples/linea...

6. BenchmarkDotNet

   5 72 10,845 8.2 C#

   Powerful .NET library for benchmarking

   

   Or use BenchmarkDotNet which, among other things to get an accurate benchmark, does JIT warmup outside of measurement.

   ( https://github.com/dotnet/BenchmarkDotNet ).

7. related_post_gen

   6 15 345 8.2 C++

   Data Processing benchmark featuring Rust, Go, Swift, Zig, Julia etc.

   

   There is one for data processing here: https://github.com/jinyus/related_post_gen

8. c-examples

   7 4 4 9.3 C

   Example C code

   

   So I actually tested your code: https://gist.github.com/bjourne/c2d0db48b2e50aaadf884e4450c6...

   On my machine single-threaded OpenBLAS multiplies two single precision 4096x4096 matrices in 0.95 seconds. Your code takes over 30 seconds. For comparison, my own matrix multiplication code (https://github.com/bjourne/c-examples/blob/master/libraries/...) run in single-threaded mode takes 0.89 seconds. Which actually beats OpenBLAS, but OpenBLAS retakes the lead for larger arrays when multi-threading is added.

9. SaaSHub

   www.saashub.com featured

   SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives

   

10. blis

    8 19 2,389 7.9 C

    BLAS-like Library Instantiation Software Framework

    

    First we can use Laser, which was my initial BLAS experiment in 2019. At the time in particular, OpenBLAS didn't properly use the AVX512 VPUs. (See thread in BLIS https://github.com/flame/blis/issues/352 ), It has made progress since then, still, on my current laptop perf is in the same range

    Reproduction:

Suggest a related project