hashtable-benchmarks VS boden

Since the blog post mentioned a PR to replace linear probing with Robin Hood, I just wanted to mention that I found bidirectional linear probing to outperform Robin Hood across the board in my Java integer set benchmarks:

https://github.com/senderista/hashtable-benchmarks/blob/mast...

https://github.com/senderista/hashtable-benchmarks/wiki/64-b...

https://homes.cs.washington.edu/~magda/papers/wang-cidr17.pd...

I'm most interested in developing high-performance database engines in low-level languages, but open to any challenging systems programming project. I've been working in C++ for the last 3 years, but have written nontrivial projects in Rust and Java as well (e.g., https://github.com/senderista/rotated-array-set, https://github.com/senderista/hashtable-benchmarks). I would enjoy using Rust or Zig on a new project, but I consider the project itself to be much more important than the language it's written in. I am not interested in cryptocurrency, adtech, or fintech projects.

Thanks for the details on your benchmarks. I would like sometime to extend BLP to a more generic setting; as I said I think any trick used with RH would also work with BLP. I just used an integer set because that's all I needed for my use case and it was easy to implement several different approaches for benchmarking. As you note, it favors use cases where the hash function is cheap (or invertible) and elements are cheap to move around.

About your question on load factors: no, the benchmarks are measuring exactly what they claim to be. The hash table constructor divides max data size by load factor to get the table size (https://github.com/senderista/hashtable-benchmarks/blob/mast...), and the benchmark code instantiates each hash table for exactly the measured data set size and load factor (https://github.com/senderista/hashtable-benchmarks/blob/mast...).

I can't explain the peaks around 1M in many of the plots; I didn't investigate them at the time and I don't have time now. It could be a JVM artifact, but I did try to use JMH "best practices", and there's no dynamic memory allocation or GC happening during the benchmark at all. It would be interesting to port these tables to Rust and repeat the measurements with Criterion. For more informative graphs I might try a log-linear approach: divide the intervals between the logarithmically spaced data sizes into a fixed number of subintervals (say 4).

I think "bidirectional linear probing" is an underrated approach (and much simpler): https://github.com/senderista/hashtable-benchmarks/blob/master/src/main/java/set/int64/BLPLongHashSet.java

I will probably never get around to porting my bidirectional linear probing integer hash set from Java to C++, but I hope someone can try adapting BLP to general C++ hashmaps and hashsets, because it significantly outperforms Robin Hood in my benchmarks.

https://homes.cs.washington.edu/~magda/papers/wang-cidr17.pd...

I'm most interested in developing high-performance database engines in low-level languages, but open to any challenging systems programming project. I've been working in C++ for the last 2 years, but have written nontrivial projects in Rust and Java as well (e.g., https://github.com/senderista/rotated-array-set, https://github.com/senderista/hashtable-benchmarks). I would enjoy using Rust or Zig on a new project, but I consider the project itself to be much more important than the language it's written in. I am not interested in cryptocurrency, adtech, or fintech projects.

Remote: Yes

Willing to relocate: Yes

Technologies: Frontend + backend, C/C++, Objective-C, TypeScript, JavaScript/ES6, Node.js, Vue/React, AWS, Python, Go, Ruby, Postgres, MySQL

Résumé/CV: on request

Email: [email protected]

Co-author of https://github.com/AshampooSystems/boden/, https://www.boden.io/

Currently COO at a SaaS startup. Prior to that, Head of Engineering at another company in the audio software industry. More than 12 years of management and development experience in numerous international projects. I have worked in both native and web development. I am a generalist with deep technical roots who is capable of quickly learning and adapting to new requirements.

Looking for a new challenge either in management or software engineering.

Location: Oldenburg, Germany

Remote: Yes

Willing to relocate: Yes

Technologies: C/C++, Objective-C, TypeScript, JavaScript/ES6, Node.js, Vue/Svelte, AWS, Python, Go, Ruby, Postgres, MySQL

Résumé/CV: on request

Email: [email protected]

Co-author of https://github.com/AshampooSystems/boden/, https://www.boden.io/

Currently COO at a startup. Prior to that, Head of Engineering at another company. More than 10 years of management and leadership experience in numerous projects, products, and circumstances.

Besides my current role, I have been working in a company group with more than 100 employees and successfully introduced Scrum and self-management methodologies in of their companies (40+ employees).

I can work as engineering manager, architect, full stack developer, or in specialized projects that require C/C++ knowledge. I have experience in nearly all fields of programming and a good ability of abstract thinking.

An example of how GPL can kill a project: Boden is a C++ GPL'd cross-platform mobile app framework that is pretty dead atm atm because of their license choice