Top 10 duplicate-file Open-Source Projects

diskover-community

15 1,399 7.7 PHP

Diskover Community Edition - Open source file indexer, file search engine and data management and analytics powered by Elasticsearch
go-find-duplicates

3 260 2.6 Go

Find duplicate files (photos, videos, music, documents) on your computer, portable hard drives etc.
InfluxDB

www.influxdata.com sponsored

Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
deduplicator

3 254 5.8 Rust

Filter, Sort & Delete Duplicate Files Recursively
dupd

1 109 0.0 C

CLI utility to find duplicate files
cbird

5 71 8.2 C++

Command-line program for managing a media collection, with focus on Content-Based Image Retrieval (Computer Vision) methods for finding duplicates.

Project mention: Similar photo finder for screenshots | /r/DataHoarder | 2023-07-04

Try cbird (my program). In addition to playing with thresholds, there are a couple of options that might help. Also if you can share the screenshots I'd be interested in testing.

dude

1 54 8.7 Python

Duplicates Detector is a cross-platform GUI utility for finding duplicate files, allowing you to delete or link them to save space. Duplicate files are displayed and processed on two synchronized panels for efficient and convenient operation. (by PJDude)

Project mention: fdupes: Identify or Delete Duplicate Files | news.ycombinator.com | 2023-11-02

Hi. I recommend my little program, the bottleneck is the gui in tkinter, but maybe it will be useful to someone:
https://github.com/PJDude/dude

detwinner

2 32 3.3 C++

Detwinner - duplicate file finder for the Linux desktop

Project mention: What are your programs missing from the official Fedora repos? | /r/Fedora | 2023-04-29

czkawka, dupeguru, and/or detwinner... if I had to pick just one then detwinner (and yes I know 2 of those are available on flathub... again, I'm lazy. And I prefer native over flatpak when possible). funny enough: I still prefer fslint to all of these... wish it was still alive

WorkOS

workos.com sponsored

The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.
duplikaatti

1 15 0.0 Go

Remove duplicate files.
samanlainen

1 6 3.0 Rust

Delete duplicate files
PSFSTools

2 3 0.0 PowerShell

Tools for File Server Role

NOTE: The open source projects on this list are ordered by number of github stars. The number of mentions indicates repo mentiontions in the last 12 Months or since we started tracking (Dec 2020).