SHOGUN VS fluxsort

The function is trying to get the median, which is not defined for an empty set. With this particular implementation, there is an assert for that:

https://github.com/shogun-toolbox/shogun/blob/9b8d85/src/sho...

Unrelatedly, but from the same section:

> Fixes are trivial, access the nth element only after the call being made. Be careful.

Wouldn't the proper fix to do the nth_element for the larget element first (for those cases that don't do that already) and then adjust the end to be the begin + larger_n for the second nth_element call? Otherwise the second call will check [begin + larger_n, end) again for no reason at all.

Steps to build a fast, highly adaptive AVX-512 sorting algorithm:

- Clone fluxsort (https://github.com/scandum/fluxsort)

- Replace the partitioning code in flux_default_partition and flux_reverse_partition with the obvious AVX-512 version using a compare and two compress instructions

- If you're feeling ambitious, swap out the small array sorting, or incorporate crumsort's fulcrum partition for larger arrays.

I know why I haven't done this: my computer doesn't have AVX-512, and hardly anyone else's seems to. Maybe a couple Zen 4 owners. I'm less clear on why the tech giants are reinventing the wheel to make these sorting alrogithms that don't even handle pre-sorted data rather than working with some of the very high-quality open source stuff out there. Is adaptivity really considered that worthless?

Fluxsort makes this particularly simple because it gets great performance out of a stable out-of-place partition. It's a bit newer; maybe the authors weren't aware of this work. But these algorithms both use (fairly difficult) in-place partitioning code; why not slot that into the well-known pdqsort?

With the right code and code-gen https://github.com/scandum/fluxsort/issues/5 these can be excellent at exploiting wide super-scalar architectures. I use them in both ipn_stable and to a larger extent in ipn_unstable which even uses the lesser known median networks for pivot selection. That said, I've done a lot of experiments with smaller sorting networks, sort3/4/5 as used in libcxx. And I found that they only look good in synthetic micro-benchmarks. If all your program does, is sort inputs of one specific size 3/4/5 in a hot loop and does nothing else, yes they are faster than insertion sort. But as soon as your application does some non-trivial amount of other work before calling sort again, the code complexity and additional branching required to get to that sort network is not worth it anymore. Depending on your architecture, my findings suggest they only start pulling ahead beginning at sizes 8-12.

Any chance you could comment on fluxsort[0], another fast quicksort? It's stable and uses a buffer about the size of the original array, which sounds like it puts it in a similar category as glidesort. Benchmarks against pdqsort at the end of that README; I can verify that it's faster on random data by 30% or so, and the stable partitioning should mean it's at least as adaptive (but the current implementation uses an initial analysis pass followed by adaptive mergesort rather than optimistic insertion sort to deal with nearly-sorted data, which IMO is fragile). There's an in-place effort called crumsort along similar lines, but it's not stable.

I've been doing a lot of work on sorting[2], in particular working to hybridize various approaches better. Very much looking forward to seeing how glidesort works.

[0] https://github.com/scandum/fluxsort

[1] https://github.com/scandum/crumsort

[2] https://mlochbaum.github.io/BQN/implementation/primitive/sor...

Fluxsort: A stable adaptive partitioning comparison sort\ (0 comments)