fdupes
dskDitto
Our great sponsors
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
fdupes
- Fdupes: Identify or Delete Duplicate Files
- fdupes: Identify or Delete Duplicate Files
-
Removing image duplicates
fdupes is simple and easy to use: https://github.com/adrianlopezroche/fdupes
- Backing Up Data: Tips/Advice for Tons of Unorganized Data and Duplicate Files from Multiple Sources
-
File Deduplication
I recently used [fdupes](https://github.com/adrianlopezroche/fdupes) to figure out duplicate files from my amazon cloud drive / photos migration. Took about 2 days to scour through about 1.5TB worth of day.
-
How would I go about copying around 5TB worth of data, from multiple drives to a singular drive/drives (Shared Pools/Raid)?
I would add the content of your current drives with rysnc to the new big drive. I would then run https://github.com/adrianlopezroche/fdupes To remove duplicate files.
- Ask HN: Tool to find identical file subtrees scattered over disks
-
Which tool do you use to find duplicate files?
jdupes, an optimized fork of the popular fdupes. There's 32-bit and 64-bit Win32 packages of jdupes there on Github.
- Mercredi Tech - 2022-05-11
- Suggestions on how to identify & report on old stale data in file shares?
dskDitto
-
Spawn multiple go routines, get the results back, in order of goroutine spawn...
Check out how I implemented my disk cleaner tool dskDitto. It might have the design pattern you’re looking for. Here is the link: https://github.com/jdefrancesco/dskDitto
-
Looking for ideas for side projects! (CPU Arch, RTL design, embedded/OS, firmware)
@OP I am working on my own x86 kernel from scratch if you’d like to help. Also I have a small utility dskDitto I’d like to actually continue developing: https://github.com/jdefrancesco/dskDitto
- Show Golang: dskDitto - Small go utility to find file duplicates rapidly
What are some alternatives?
rdfind - find duplicate files utility
duplikaatti - Remove duplicate files.
jdupes - A powerful duplicate file finder and an enhanced fork of 'fdupes'.
xxhash - A Go implementation of the 64-bit xxHash algorithm (XXH64)
rmlint - Extremely fast tool to remove duplicates and other lint from your filesystem
goneric - Go generic utilities
czkawka - Multi functional app to find duplicates, empty folders, similar images etc.
goreleaser - Deliver Go binaries as fast and easily as possible
go-find-duplicates - Find duplicate files (photos, videos, music, documents) on your computer, portable hard drives etc.
Telegraf - The plugin-driven server agent for collecting & reporting metrics.
dupeguru - Find duplicate files
fclones - Efficient Duplicate File Finder