dupd
fclones
| dupd | fclones | |
|---|---|---|
| 1 | 20 | |
| 114 | 2,376 | |
| 0.0% | 3.0% | |
| 0.0 | 6.2 | |
| about 2 years ago | 4 months ago | |
| C | Rust | |
| GNU General Public License v3.0 only | MIT License |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
dupd
-
Go Find Duplicates: blazingly-fast simple-to-use tool to find duplicate files
I use and test assorted duplicate finders regularly.
fdupes is the classic (going way way back) but it's really very slow, not worth using anymore.
The four I know are worth trying these days (depending on data set, hardware, file arrangement and other factors, any one of these might be fastest for a specific use case) are https://github.com/jbruchon/jdupes , https://github.com/pauldreik/rdfind , https://github.com/jvirkki/dupd , https://github.com/sahib/rmlint
Had not encountered fclones before, will give it a try.
fclones
-
Hyperspace
I've been using `fclones` [1] to do this, with `dedupe`, which uses reflink/clonefile.
https://github.com/pkolaczk/fclones
- Fclones: Parallel duplicate files finder in Rust with reflink support
- Fclones: Efficient Duplicate File Finder
- fdupes: Identify or Delete Duplicate Files
-
Deduplication at file level
Install fclones (fdupes modern equivalent)
- A duplicate file finder with GUI written using gtk4-rs and relm4
-
making duplicate files hard links, probably rdfind
I said "fclone" I meant "fclones" (https://github.com/pkolaczk/fclones) -- I missed the "s" at the end
-
Use `dano` to find duplicate media files
While we're recommending stuff, fclones: https://github.com/pkolaczk/fclones
-
Oxidise Your Life
fclones: Efficient Duplicate File Finder.
-
Pip and cargo are not the same
Yes, that would be safer when available (although generally files within library dependencies are not modified I think?). It looks like fclones implements this, is faster and is written in Rust https://github.com/pkolaczk/fclones (the last is the most important point of course /s).
What are some alternatives?
mpifileutils - File utilities designed for scalability and performance.
czkawka - Multi functional app to find duplicates, empty folders, similar images etc.
duphard - A simple utility to detect duplicate files and replace them with hard links.
rmlint - Extremely fast tool to remove duplicates and other lint from your filesystem
go-find-duplicates - Find duplicate files (photos, videos, music, documents) on your computer, portable hard drives etc.
rdfind - find duplicate files utility