FastHash VS pyroscope-rs

I've implemented your RecSplit method in my MPHF benchmark library (written in C#). The suite is not yet public, but I do want to say thank you for your fantastic method/code/paper. In my own rabbithole research, I stumbled upon several artifacts of your rabbithole trail. Most notably the stuff on StackOverflow, which helped my own research.

I've releaed a set og fast hash functions[1] to help gain an understanding of speed vs. quality. My biggest takeaway is that most generic hash functions can be specialized for integer inputs[2], which often reduce latency by quite a lot, making MPFH more attractive over simple iteration on small sets, as the overhead of hashing is considerably smaller.

[1] https://github.com/Genbox/FastHash

[2] https://github.com/Genbox/FastHash/blob/master/src/FastHash/...

It was made (by Sanmayce) to optimize for instruction-level pipelining, and use the fact that modern CPUs have multiple execution ports. But due to those changes, it is not compatible with FNV1a anymore.

The trick of reading in stripes is employed by many of the fastest hashes. It is kinda funny to see how one author prefers a switch case over for loops, where others prefer while loops. The differences can sometimes have a big impact on what optimizations the compiler decides to use.

[1] https://github.com/Genbox/FastHash/blob/master/src/FastHash....

At the moment I'm working on FastHash[1], a pet project of mine to port a few high-performance non-cryptographic hash functions to C#.

I'm also trying to build FastLinq, a value-by-reference Language Integrated Query (LINQ) optimized for high-performance scenarios. It is kind of a weird mix as LINQ in .NET is known for its high overhead.

Finally, I'm working on an Office setting synchronization application. I heard a podcast with Paul Thurrott complaining about the lack of sync solutions, so I thought I would do one for fun.

[1] https://github.com/Genbox/FastHash

A general purpose profiler: https://github.com/pyroscope-io/pyroscope-rs

If someone is interested in this space, feel free to reach me!

I feel some of the OP points. I was working on a profiling agent lately, and one of the issues was running it on multiple platforms (just the four big ones linux/mac-x86/arm) on FFI (because it'll be run directly from python/ruby/etc...) and preferably having the thing just work without having to install or configure any dependencies.

Like OP I hit two walls: libunwind, and linking. For libunwind, I ended up downloading/compiling manually; and for linking there is auditwheel[1]. Although it is a Python tool, I did actually end up using for Ruby (by creating a "fake python package", and then copying the linked dependencies).

It was at that time that I learned about linking for dynamic libraries, patchelf and there is really no single/established tool to do this. I thought there should be something but most people seem to install the dependencies with any certain software. I also found, the hard way, that you still have to deal with gcc/c when working with Rust. It does isolate you from many stuff, but for many things there is no work around.

There is a performance hit to this strategy, however, since shared dynamic libraries will be used by all the running programs that need them; whereas my solution will run its own instance. It made me wonder if wasm will come up with something similar without affecting portability.

Finally, the project is open source and you can browse the code here: https://github.com/pyroscope-io/pyroscope-rs

[1]: https://github.com/pypa/auditwheel

Version 0.5 is now live!: https://github.com/pyroscope-io/pyroscope-rs

Working on https://github.com/pyroscope-io/pyroscope-rs A profiling solution for Rust and other languages.

Thanks to the maintainers at pprof-rs for helping us figure out how we can modify their profiler to create our rust agent (https://github.com/pyroscope-io/pyroscope-rs).