dduper VS dupeguru

In some cases deduplication happens at the file system layer transparently without you even realizing it. E.g. there are tools like https://github.com/lakshmipathi/dduper

I agree that image editing workflows are a different use case more suited to perceptual hashes than cryptographic hashes.

ddupper uses a patched btrfs command to read file hases from the raw disk. It requires root access and is kind of a hack.

There are various tools to use for COW deduplication, such as bees, duperemove, rmlint, jdupes, and dduper

Hmm, could we use something like dduper to achieve this, if they don't have it included?

Dupeguru

Posted link appears be misspelled

This appears to be actual link:

https://dupeguru.voltaicideas.net/

I used to use DupeGuru which has some photo-specific dupe detection where you can fuzzy match image dupes based on content: https://dupeguru.voltaicideas.net/

But I switched over to czkawka, which has a better interface for comparing files, and seems to be a bit faster: https://github.com/qarmin/czkawka

Unfortunately, neither of these are integrated into Photoprism, so you still have to do some file management outside the database before importing.

I also haven't used Photoprism extensively yet (I think it's running on one of my boxes, but I haven't gotten around to setting it up), but I did find that it wasn't really built for file-based libraries. It's a little more heavyweight, but my research shows that Nextcloud Memories might be a better choice for me (it's not the first-party Nextcloud photos app, but another one put together by the community): https://apps.nextcloud.com/apps/memories

If your photos are exact duplicates you can also use the freeware dupeGuru app.

If you want a some other GUI options, AllDup or DupeGuru might be of interest. AllDup has a rather weird interface, though, imo.

I used DupeGuru (https://dupeguru.voltaicideas.net/) in the past but I'm not sure it's the best solution for you. Try it, it's open-source.

dupeGuru

dupeGuru is a graphical application available for macOS that can scan multiple drives and identify duplicate files based on various criteria like name, size, and content https://dupeguru.voltaicideas.net/