PSI
RoaringBitmap
PSI | RoaringBitmap | |
---|---|---|
3 | 24 | |
125 | 3,390 | |
0.0% | 0.9% | |
5.2 | 8.5 | |
25 days ago | 15 days ago | |
C++ | Java | |
Apache License 2.0 | Apache License 2.0 |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
PSI
-
Can a new form of cryptography solve the internet’s privacy problem?
There are other techniques that aren't generally included in the "Zero Knowledge Proofs" set of techniques that are perhaps more practical for general development.
For example, I fine private set intersection[1] as implemented by OpenMined a really useful primative a bunch of privacy enhancing applications can be built on top of.
My colleagues and I recently published a pre-print[2] showing how to use this for sharing locations you and another person have had in common, without being able to see other locations. The paper talks about a social network built around this but I also think there are useful applications in things like real-world games (scavenger hunts etc)
[1] https://github.com/OpenMined/PSI/blob/master/private_set_int...
[2] https://arxiv.org/abs/2210.01927
-
Ask HN: What are some 'cool' but obscure data structures you know about?
I came here to say Golomb compressed sets except now I see it's part of the question!
They are used by default in the OpenMined implementation of Private Set Intersection[1] - a multi-party computation technique.
[1] https://github.com/OpenMined/PSI/blob/master/private_set_int...
-
Is there a Private Set Intersection protocol where the server learns the length of the intersection?
I was using OpenMinded/PSI exploring some PSI implementations, but I would like a way for the server to know the intersection size. Say Signal wants to calculate the average number of users from one person's address book (or whatever).
RoaringBitmap
-
Iterating over Bit Sets Quickly
I was recently reading about Roaring https://roaringbitmap.org/ which is a highly optimized compressed bitset implementation. I reccomend reading about it if you are interested in this sort of thing. The talk at https://roaringbitmap.org/talks/ is especially good.
- Roaring Bitmaps
- Roaring bitmaps are compressed bitmaps, can be 100x faster
-
What feature would you like to remove in C++26?
However, I would love compressed (not just packed) bitsets too, which is something different to me. I would make it another class with a similar interface, based on something like roaring. It doesn't need to be in the standard, but it would be nice if the API was a such that one could easily swap implementations.
-
Jaccard Index
As an aside if you find yourself having to compute them on the fly, know that the Roaring Bitmaps libraries is the way to go [1]. The bitmaps are compressed, and can be streamed directly into SIMD computations (batching XORs and popcnts 256 bits wide!). The Jaccard index is just intersection_len / union_len [2] away
[1] https://roaringbitmap.org/
[2] https://roaringbitmap.readthedocs.io/en/latest/#roaringbitma...
-
Looking for fast, space-efficient key-lookup
Use a two stage approach, with a bloom/cuckoo filter stored as a https://roaringbitmap.org/ in memory. Then a secondary key/value store on disk (bolt or anything else).
-
BitSet Vs BigInteger
As an aside, if you're dealing with large bit sets, you might also want to evaluate Roaring Bitmaps.
-
Negative Incentives in Academic Research
Sidetracking a bit the conversation. What a coincidence that the author (Lemire) is also represented on Today's #1 "Ask HN: What are some cool but obscure data structures you know about?" as he is the main contributor of RoaringBitmap https://github.com/RoaringBitmap/RoaringBitmap and one of the main authors of the data structure.
- Ask HN: What are some 'cool' but obscure data structures you know about?
- Roaring bitmaps: A better compressed bitset
What are some alternatives?
ctrie-java - Java implementation of a concurrent trie
HyperMinHash-java - Union, intersection, and set cardinality in loglog space
AspNetCoreDiagnosticScenarios - This repository has examples of broken patterns in ASP.NET Core applications
lucene - Apache Lucene open-source search software
t-digest - A new data structure for accurate on-line accumulation of rank-based statistics such as quantiles and trimmed means
CQEngine - Ultra-fast SQL-like queries on Java collections
cheerp-meta - Cheerp - a C/C++ compiler for Web applications - compiles to WebAssembly and JavaScript
Primes - Prime Number Projects in C#/C++/Python
swift - the multiparty transport protocol (aka "TCP with swarming" or "BitTorrent at the transport layer")
Feign - Feign makes writing java http clients easier
pvfmm - A parallel kernel-independent FMM library for particle and volume potentials
maven-compiler-plugin - Apache Maven Compiler Plugin