Halide VS TensorOperations.jl

If CPU/GPU execution speed is the goal while simultaneously code golfing the source size, https://halide-lang.org/ might have come in handy.

This is a task where Halide https://halide-lang.org/ could really shine! It disconnects logic from scheduling (unrolling, vectorizing, tiling, caching intermediates etc), so every step the author describes in the article is a tunable in halide. halide doesn't appear to have bindings for golang so calling C++ from go might be the only viable option.

Probably would have been much easier to do 15 times in https://halide-lang.org/

The idea behind Halide is that scheduling memory access patterns is critical to performance. But, access patterns being interwoven into arithmetic algorithms makes them difficult to modify separately.

So, in Halide you specify the arithmetic and the schedule separately so you can rapidly iterate on either.

It is not the sorting per-se which was improved here, but sorting (particularly short sequences) on modern CPUs with really the complexity being on the difficulty of predicting what will work quickly on these modern CPUs.

Doing an empirical algorithm search to find which algorithms fit well on modern CPUs/memory systems is pretty common, see e.g. FFTW, ATLAS, https://halide-lang.org/

Halide https://halide-lang.org/

It doesn’t apply in this case, but in general if you really want the best vectorization I would suggest using https://halide-lang.org instead of trying to coerce your compiler.

Sometimes broadcasting feels like magic to me. It just works more often than not even when I am confused with the dimensions. If you do a lot of Tensor stuff it's also worth checking out Einstein notation (https://github.com/Jutho/TensorOperations.jl)

There are some libraries and macros for Einstein notation and related ideas, like TensorOperations.jl in Julia, einsum in numpy which someone already mentioned, and some small-scale/research languages like Diderot and Egison. In the mainstream, I guess languages generally use for loops or list comprehensions and try to recover vectorisation from that after the fact, but don’t guarantee it. Those that do make guarantees tend to use combinators that are matrixwise/function-level. I admit I pretty much categorically prefer the latter so I’m not as aware of the state of this as I’d like to be able to help.