dupd
czkawka
| dupd | czkawka | |
|---|---|---|
| 1 | 365 | |
| 114 | 24,418 | |
| 0.0% | 2.7% | |
| 0.0 | 7.4 | |
| about 2 years ago | 2 months ago | |
| C | Rust | |
| GNU General Public License v3.0 only | GNU General Public License v3.0 or later |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
dupd
-
Go Find Duplicates: blazingly-fast simple-to-use tool to find duplicate files
I use and test assorted duplicate finders regularly.
fdupes is the classic (going way way back) but it's really very slow, not worth using anymore.
The four I know are worth trying these days (depending on data set, hardware, file arrangement and other factors, any one of these might be fastest for a specific use case) are https://github.com/jbruchon/jdupes , https://github.com/pauldreik/rdfind , https://github.com/jvirkki/dupd , https://github.com/sahib/rmlint
Had not encountered fclones before, will give it a try.
czkawka
-
Umami is a simple, fast, privacy-focused alternative to Google Analytics
There's Czkawka, Polish for "hiccup" application that helps find duplicates [1], dev behind it also created files renamer Szyszka, again "cone" in Polish [2]. Backup solution Kopia [3] mean "copy" but also "lance" and "spear" hence the pointy thing in its logo - if I recall correctly one of devs is Polish.
[1] - https://github.com/qarmin/czkawka
-
Ask HN: How do you deduplicate files?
You want content-addressed storage; this works with rolling content hashes that identify common blocks of memory. `rsync` uses that technique to minimize bytes to be transferred. https://github.com/qarmin/czkawka is a GUI app and CLI tool to find identical files in general and similar images in particular.
The task is much simpler if you only want to find bit-identical entire files, not part of files; in that case, you can just run a tool like `sha1sum` over each file and record the hash digest in a database; identical files—and only identical ones, with high probability—will have the same hash, non-identical ones will have different hashes.
- Czkawka: Multi functional app to find duplicates, empty folders, similar images
-
Duperemove – Tools for deduping file systems
You might be interested in this app: https://github.com/qarmin/czkawka
- Is there software to compress large but similar files?
- Merge three separate partial libraries from external USB drives
-
Tools to deduplicate files
https://github.com/qarmin/czkawka by far the best of anything iv tried
-
fdupes: Identify or Delete Duplicate Files
I've used Czkawka (https://github.com/qarmin/czkawka) because it does Lanczos-based image duplicate detection, which makes it more practical for me.
-
AllDup suddenly taking forever to process/delete selections
Maybe it's a setting you made or the files, not sure. You can try another software czkawka to see if you get better results with it.
-
Is there a file duplicate finder that works with animated jpegxl-gif?
For static images i used https://github.com/qarmin/czkawka and it works well enough. I think. But when i used it on a folder with gifs and their jxl conversions, it shows nothing. SURELY this could not be user error, rrrright?
What are some alternatives?
mpifileutils - File utilities designed for scalability and performance.
dupeguru - Find duplicate files
fclones - Efficient Duplicate File Finder
fdupes - FDUPES is a program for identifying or deleting duplicate files residing within specified directories.
duphard - A simple utility to detect duplicate files and replace them with hard links.
bleachbit - BleachBit system cleaner for Windows and Linux