dupd
duphard
| dupd | duphard | |
|---|---|---|
| 1 | 1 | |
| 109 | 2 | |
| - | - | |
| 0.0 | 0.0 | |
| 11 months ago | almost 4 years ago | |
| C | Go | |
| GNU General Public License v3.0 only | GNU General Public License v3.0 only |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
dupd
-
Go Find Duplicates: blazingly-fast simple-to-use tool to find duplicate files
I use and test assorted duplicate finders regularly.
fdupes is the classic (going way way back) but it's really very slow, not worth using anymore.
The four I know are worth trying these days (depending on data set, hardware, file arrangement and other factors, any one of these might be fastest for a specific use case) are https://github.com/jbruchon/jdupes , https://github.com/pauldreik/rdfind , https://github.com/jvirkki/dupd , https://github.com/sahib/rmlint
Had not encountered fclones before, will give it a try.
duphard
-
Go Find Duplicates: blazingly-fast simple-to-use tool to find duplicate files
For example I maintain a tar file and a docker image with Kafka connectors which share many jar files. Using duphard I can save hundreds of megabytes, or even more than a gigabyte! For a documentation website with many copies of the same image (let's just say some static generators favor this practice for maintaining multiple versions), I can reduce the website size by 60%+, which then makes ssh copies, docker pulls, etc way faster speeding up deployment times.
https://github.com/andmarios/duphard
What are some alternatives?
fclones - Efficient Duplicate File Finder
rdfind - find duplicate files utility
jdupes - A powerful duplicate file finder and an enhanced fork of 'fdupes'.
mpifileutils - File utilities designed for scalability and performance.
go-find-duplicates - Find duplicate files (photos, videos, music, documents) on your computer, portable hard drives etc.
fdupes - FDUPES is a program for identifying or deleting duplicate files residing within specified directories.
rmlint - Extremely fast tool to remove duplicates and other lint from your filesystem
czkawka - Multi functional app to find duplicates, empty folders, similar images etc.