janusgraph VS xxHash

First, you need to choose a specific graph database platform to work with, such as Neo4j, OrientDB, JanusGraph, Arangodb or Amazon Neptune. Once you have selected a platform, you can then start working with graph data using the platform's query language.

JanusGraph

There were a couple years where it was mostly abandoned. Good to see this solid graph database being well maintained. the milestones[1] mostly show a lot of upgrading libraries, some enhancements/features sprinkled in, but for a while Janus was nearly abandoned.

Maintenance re-started in 2017, with IBM & Google stepping up to back it[2].

[1] https://github.com/JanusGraph/janusgraph/milestones?state=cl...

[2] https://architecht.io/google-ibm-back-new-open-source-graph-...

QOMPLX partnered with the graph database experts at Expero to implement their system with JanusGraph, which uses Scylla as an underlying fast and scalable storage layer. We had the privilege to learn from their use case at Scylla Summit this January, which we share with you today.

At the time of writing this article, JanusGraph's latest release page on GitHub says that JanusGraph is compatible with Cassandra 3.11.0. So I will get this version's download link from the Apache Cassandra archive, and I will use the wget shell command to download it to my /opt directory. Then I will use the tar command to extract the contents of the downloaded archive.

> GPU Hash Table?

How bad would performance have suffered if you sha256'd the lines to build the map? I'm going to guess "badly"?

Maybe something like this in CUDA: https://github.com/Cyan4973/xxHash ?

Examples of fast, simple hashes that are independent enough includes murmur, xxHash, Fowler–Noll–Vo hash function and many others

If you're only using the hash for non-cryptographic applications, there are much faster hashes: https://github.com/Cyan4973/xxHash

I found the benchmark result of AMD ryzen 5950X

But what's the catch, you're thinking ? Well, it is a bit slower than its counterparts when it comes to deserializing (and marginally faster for serializing). To achieve smaller footprint, it uses a few tricks and notably a custom hash table to deduplicate strings. This comes at a cost of course (even when featuring xxHash to speed things up), but keeps the slowdown reasonable (I think).

Non cryptographic hashes has collisions, for example, assume you having content like "abcdefg" which hashed value is "123", in case of weak hash algorithm some other content like "abcdefZ" can also have a hash "123" which basically means such hash function is failed to be unique fingerprint of particular content. BLAKE3 for example can do 6-7Gb/s which make it pretty fast and secure. If your requirement accepts collision with defined error rate, I would advise you to take a look at XXH3 if you need very snappy hash algorithm, which can run at pace or RAM access (30GB/s+), but again, run tests at particular equipment you targeting, may be AES hardware accelerated MeowHash will serve you better.