Top 7 Python similarity Projects

fastdup

18 1,403 9.4 Python

fastdup is a powerful free tool designed to rapidly extract valuable insights from your image & video datasets. Assisting you to increase your dataset images & labels quality and reduce your data operations costs at an unparalleled scale.

Project mention: Visualize your dataset using DINOv2 embedding | news.ycombinator.com | 2023-05-02

Visualizing your dataset (especially large ones) in a low-dimensional embedding space can tell you a lot about the patterns and clusters in your dataset.
We recently release a notebook showing how you can visualize your dataset using DINOv2 models by running it on your CPU.
Yes! No GPUs needed.
We used it to find clusters of similar images, duplicates, and outliers in a subset of the LAION dataset
Try it on your own dataset:
Colab notebook - https://colab.research.google.com/github/visual-layer/fastdup/blob/main/examples/dinov2_notebook.ipynb
GitHub repo - https://github.com/visual-layer/fastdup

python-string-similarity

1 946 0.0 Python

A library implementing different string similarity and distance measures using Python.
InfluxDB

www.influxdata.com sponsored

Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
Duplicate-Image-Finder

3 390 9.0 Python

difPy - Python package for finding duplicate or similar images within folders
aurora

1 74 0.0 Python

Malware similarity platform with modularity in mind. (by W3ndige)
unisim

1 63 8.1 Python

UniSim is a package for efficient similarity computation, fuzzy matching, and clustering of data.

Project mention: Google UniSim for efficient similarity computation | news.ycombinator.com | 2023-11-30

pysimilar

1 19 0.0 Python

A python library for computing the similarity between two strings (text) based on cosine similarity
image-deduplication-plugin

1 8 5.7 Python

Remove exact and approximate duplicates from your dataset in FiftyOne!

Project mention: Plugin for Building and Managing Plugins! | dev.to | 2024-02-09

Week 4: 🪞Image Deduplication

WorkOS

workos.com sponsored

The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.

NOTE: The open source projects on this list are ordered by number of github stars. The number of mentions indicates repo mentiontions in the last 12 Months or since we started tracking (Dec 2020).