fdupes
kopia
Our great sponsors
fdupes | kopia | |
---|---|---|
17 | 224 | |
2,354 | 6,270 | |
- | 5.8% | |
2.5 | 9.6 | |
about 2 months ago | 6 days ago | |
C | Go | |
- | Apache License 2.0 |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
fdupes
- Fdupes: Identify or Delete Duplicate Files
- fdupes: Identify or Delete Duplicate Files
-
Removing image duplicates
fdupes is simple and easy to use: https://github.com/adrianlopezroche/fdupes
- Backing Up Data: Tips/Advice for Tons of Unorganized Data and Duplicate Files from Multiple Sources
-
File Deduplication
I recently used [fdupes](https://github.com/adrianlopezroche/fdupes) to figure out duplicate files from my amazon cloud drive / photos migration. Took about 2 days to scour through about 1.5TB worth of day.
-
How would I go about copying around 5TB worth of data, from multiple drives to a singular drive/drives (Shared Pools/Raid)?
I would add the content of your current drives with rysnc to the new big drive. I would then run https://github.com/adrianlopezroche/fdupes To remove duplicate files.
- Ask HN: Tool to find identical file subtrees scattered over disks
-
Which tool do you use to find duplicate files?
jdupes, an optimized fork of the popular fdupes. There's 32-bit and 64-bit Win32 packages of jdupes there on Github.
- Mercredi Tech - 2022-05-11
- Suggestions on how to identify & report on old stale data in file shares?
kopia
-
DwarFS – The Deduplicating Warp-Speed Advanced Read-Only File System
I think Kopia would be great for your use case
https://kopia.io/
It has a great system to snapshot files but only store data if it's changed. I use it in an environment where I can't use something like zfs to snapshot data because I don't have the ability to make decisions about what filesystem we're using. It's been amazing, love it so much!
-
Ask HN: Open-source Windows 11 backup solutions
Thanks for the tip on Kopia. Setting it up now, looks perfect.
https://github.com/kopia/kopia
-
Building a Managed Service Provider Business With Open Source
Kopia - GitHub
-
I Backup
I've been happy with: https://kopia.io/
Fairly easy to configure, does snapshots to S3 and has a icon in my tray I can watch :)
-
Very strange behavior/bug - devices stuck together
Btw, kopia is one fine backup tool. Apparently borgbackup is good too.
-
Kopia: Open-Source, Fast and Secure Open-Source Backup Software
Kopia is great, though it's worth noting for folks on Linux: non-UTF-8 paths aren't stored correctly [1] and xattrs aren't stored [2]. While most folks probably won't care about the former, the latter can could cause issues (eg. losing SELinux labels makes it difficult to restore a backup of the root filesystem on distros that use SELinux).
[1] https://github.com/kopia/kopia/issues/1764
[2] https://github.com/kopia/kopia/issues/544
- Kopia: Open-Source Backup Software
-
How I backup my servers (2023)
I think Kopia [1] is on its way to be that. I am sticking to Restic for now but it seems like the strongest contender.
[1]: https://github.com/kopia/kopia
-
Borgbase backups have been unavailable for 3 days – recovery is at 26%
I used their trial for a bit to test it out with Vorta [1] in a container. Vorta (and Borg) seemed to work fine, until I wanted to restore an archive and I noticed that my recent snapshots were completely empty. Probably because of a misconfiguration on my end though. But it made me look elsewhere. For me backups should be a fire, test and forget solution.
Recently I made the switch to Kopia [2] which seems to have feature parity with Borg (and Restic [3]). It also has a web UI which is way easier to work with than Vorta. And I can easily view, extract and restore individual files or folders from there. This gave me way more confidence about this solution. The only thing I really miss is that I cannot chose different targets for different paths. For instance, with Borg I was able to backup a partial of my Docker appdata to an external source. And I haven't found a way to do this with Kopia. Besides that I'm pretty happy with this solution and I would recommend it.
1. https://vorta.borgbase.com/
2. https://kopia.io/
3. https://restic.net/
-
Show HN: Gdańsk AI – full stack AI voice chatbot (STT, LLM, TTS, auth, payments)
There's a few. Off the top of my head
https://github.com/kopia/kopia
What are some alternatives?
rdfind - find duplicate files utility
restic - Fast, secure, efficient backup program
jdupes - A powerful duplicate file finder and an enhanced fork of 'fdupes'.
BorgBackup - Deduplicating archiver with compression and authenticated encryption.
rmlint - Extremely fast tool to remove duplicates and other lint from your filesystem
Duplicati - Store securely encrypted backups in the cloud!
czkawka - Multi functional app to find duplicates, empty folders, similar images etc.
UrBackup - UrBackup - Client/Server Open Source Network Backup for Windows, MacOS and Linux
go-find-duplicates - Find duplicate files (photos, videos, music, documents) on your computer, portable hard drives etc.
rclone - "rsync for cloud storage" - Google Drive, S3, Dropbox, Backblaze B2, One Drive, Swift, Hubic, Wasabi, Google Cloud Storage, Yandex Files
dupeguru - Find duplicate files
restic-wrapper - Simple bash wrapper to source .env configuration files for Restic. Facilitates both manual CLI execution and scheduled (cron) execution.