fdupes
czkawka
Our great sponsors
| fdupes | czkawka | |
|---|---|---|
| 17 | 361 | |
| 2,354 | 17,429 | |
| - | - | |
| 2.5 | 7.9 | |
| about 2 months ago | 26 days ago | |
| C | Rust | |
| - | GNU General Public License v3.0 or later |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
fdupes
- Fdupes: Identify or Delete Duplicate Files
- fdupes: Identify or Delete Duplicate Files
-
Removing image duplicates
fdupes is simple and easy to use: https://github.com/adrianlopezroche/fdupes
- Backing Up Data: Tips/Advice for Tons of Unorganized Data and Duplicate Files from Multiple Sources
-
File Deduplication
I recently used [fdupes](https://github.com/adrianlopezroche/fdupes) to figure out duplicate files from my amazon cloud drive / photos migration. Took about 2 days to scour through about 1.5TB worth of day.
-
How would I go about copying around 5TB worth of data, from multiple drives to a singular drive/drives (Shared Pools/Raid)?
I would add the content of your current drives with rysnc to the new big drive. I would then run https://github.com/adrianlopezroche/fdupes To remove duplicate files.
- Ask HN: Tool to find identical file subtrees scattered over disks
-
Which tool do you use to find duplicate files?
jdupes, an optimized fork of the popular fdupes. There's 32-bit and 64-bit Win32 packages of jdupes there on Github.
- Mercredi Tech - 2022-05-11
- Suggestions on how to identify & report on old stale data in file shares?
czkawka
- Is there software to compress large but similar files?
- Merge three separate partial libraries from external USB drives
-
Tools to deduplicate files
https://github.com/qarmin/czkawka by far the best of anything iv tried
-
fdupes: Identify or Delete Duplicate Files
I've used Czkawka (https://github.com/qarmin/czkawka) because it does Lanczos-based image duplicate detection, which makes it more practical for me.
-
AllDup suddenly taking forever to process/delete selections
Maybe it's a setting you made or the files, not sure. You can try another software czkawka to see if you get better results with it.
-
Is there a file duplicate finder that works with animated jpegxl-gif?
For static images i used https://github.com/qarmin/czkawka and it works well enough. I think. But when i used it on a folder with gifs and their jxl conversions, it shows nothing. SURELY this could not be user error, rrrright?
-
PhotoPrism: Browse Your Life in Pictures
I used to use DupeGuru which has some photo-specific dupe detection where you can fuzzy match image dupes based on content: https://dupeguru.voltaicideas.net/
But I switched over to czkawka, which has a better interface for comparing files, and seems to be a bit faster: https://github.com/qarmin/czkawka
Unfortunately, neither of these are integrated into Photoprism, so you still have to do some file management outside the database before importing.
I also haven't used Photoprism extensively yet (I think it's running on one of my boxes, but I haven't gotten around to setting it up), but I did find that it wasn't really built for file-based libraries. It's a little more heavyweight, but my research shows that Nextcloud Memories might be a better choice for me (it's not the first-party Nextcloud photos app, but another one put together by the community): https://apps.nextcloud.com/apps/memories
-
Please don't post like 20 similar images to the art sites?
Czkawka can do this.
-
I'm amazed how I find anything & why I have so many dupes!
There's always the well-respected tool, Czkawka. Or, of the CLI is your thing, jdupes is a good option.
- I saw a post regarding crate to delete similar files
What are some alternatives?
rdfind - find duplicate files utility
dupeguru - Find duplicate files
jdupes - A powerful duplicate file finder and an enhanced fork of 'fdupes'.
rmlint - Extremely fast tool to remove duplicates and other lint from your filesystem
AntiDupl - A program to search similar and defect pictures on the disk
go-find-duplicates - Find duplicate files (photos, videos, music, documents) on your computer, portable hard drives etc.
PhotoPrism - AI-Powered Photos App for the Decentralized Web 🌈💎✨
darktable - darktable is an open source photography workflow application and raw developer
fclones - Efficient Duplicate File Finder
datacurator-filetree - a standard filetree for /r/datacurator [ and r/datahoarder ]