Top 5 Python Minhash Projects

datasketch

1 2,348 6.4 Python

MinHash, LSH, LSH Forest, Weighted MinHash, HyperLogLog, HyperLogLog++, LSH Ensemble and HNSW
sourmash

1 432 9.4 Python

Quickly search, compare, and analyze genomic and metagenomic data sets.
InfluxDB

www.influxdata.com sponsored

Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
LSH

1 273 2.8 Python

Locality Sensitive Hashing using MinHash in Python/Cython to detect near duplicate text documents
intertext

1 110 0.0 Python

Detect and visualize text reuse

Project mention: Ask HN: Tool to find text reuse, similar paragraphs, fuzzy/near dupes in folder? | news.ycombinator.com | 2023-06-25

Do you know of any too that I can use to compare my own notes and documents vault in search for copied paragraphs or almost similar phrases? Normal diffing/hashing wouldn't work as we're talking about the contents of slightly modified documents, and the comparison of each file against all others.
I found the following tools that seem related yet not quite there, maybe I'm missing a particular term of art?
https://github.com/YaleDHLab/intertext

Neural-Scam-Artist

2 22 0.0 Python

Web Scraping, Document Deduplication & GPT-2 Fine-tuning with a newly created scam dataset.
WorkOS

workos.com sponsored

The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.

NOTE: The open source projects on this list are ordered by number of github stars. The number of mentions indicates repo mentiontions in the last 12 Months or since we started tracking (Dec 2020).