datasketch
intertext
datasketch | intertext | |
---|---|---|
1 | 1 | |
2,352 | 110 | |
- | 0.0% | |
6.4 | 0.0 | |
about 1 month ago | 12 months ago | |
Python | Python | |
MIT License | - |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
datasketch
-
D Efficient Way To Cluster Millions Of Face
A great library that implements all those "big data" algos is https://github.com/ekzhu/datasketch
intertext
-
Ask HN: Tool to find text reuse, similar paragraphs, fuzzy/near dupes in folder?
Do you know of any too that I can use to compare my own notes and documents vault in search for copied paragraphs or almost similar phrases? Normal diffing/hashing wouldn't work as we're talking about the contents of slightly modified documents, and the comparison of each file against all others.
I found the following tools that seem related yet not quite there, maybe I'm missing a particular term of art?
https://github.com/YaleDHLab/intertext
What are some alternatives?
image-ndd-lsh - Near-duplicate image detection using Locality Sensitive Hashing
sourmash - Quickly search, compare, and analyze genomic and metagenomic data sets.
LSH - Locality Sensitive Hashing using MinHash in Python/Cython to detect near duplicate text documents
neardup - Near-duplicate detection
dedup - Find duplicate text files.
dash-tools - DashTools - Plotly Dash Command Line Tools - Create, Run and Deploy Templated Python Apps from Terminal
searx - Privacy-respecting metasearch engine [Moved to: https://github.com/searx/searx]
Neural-Scam-Artist - Web Scraping, Document Deduplication & GPT-2 Fine-tuning with a newly created scam dataset.
jina - ☁️ Build multimodal AI applications with cloud-native stack
dash - Data Apps & Dashboards for Python. No JavaScript Required.