mmperf VS Flops

Here are the matmul sizes for the MiniLM model used for inference: https://github.com/mmperf/mmperf/blob/main/benchmark_sizes/b...

These are the matmul sizes for the BERT training workload https://github.com/mmperf/mmperf/blob/main/benchmark_sizes/b...

Yes we use the latest MoltenVK (1.3.204.0) installed in the system.

I will let @noxa and other IREE devs chime in on the SPIR-V path but we do support prefix sums etc in the GPU path.

//part of nod.ai team.

Could you give me a benchmark in particular? Or maybe this one works: https://github.com/mmperf/mmperf. I'll run it in an hour.

Go and measure it yourself, if you have one :)

https://github.com/Mysticial/Flops/

You can also get a theoretical computation of the Flops, which matches nicely with the experimental measurement. You have to take into account:

- the clock frequency (~3.9 GHz on multithreaded workloads on my machine)

In synthetic tests (that are probably 1-2% lower than theoretical max FLOPS), my i9-7940x consistently pulls around 1525 GFLOPS DP / 3050 GFLOPS SP (14 core, 3.6 GHz, AVX512). Admittedly, my CPU is overclocked, but not that much...for AVX512 stuff mine goes 3.6 GHz vs 3.1 GHz base speed. Extrapolating, at base clock this is 1313 GFLOPS DP / 2626 GFLOPS SP. This is ~3.2024x (or a +220.24% increase) Broadwell-e.

That said, I am pushing nearly 2 TFLOPS compute (this is real world compute - synthetic benchmarks like this top out at ~3.3 TFLOPS), so I suppose that is fair lol.