imagededup VS czkawka

imagededup

To generate the hashes I was just going to use https://github.com/idealo/imagededup as it has been effective in my testing. The issue is the scalability issue. That code even is able to find similar images, using distance hamming. But they didn't care about scalability, and that is now my biggest bottleneck.

Which is why, it's much better to use a dedicated tool that can visually compare photos (using a neural net/machine learning model) to identify duplicate images. If you want a command-line tool for scripting purposes then check out imgdup2go or imagededup.

I had used phash to find the duplicate images. The hashing algorithm is ingenious...And it was able to find duplicates (and thus remove) efficiently imagededub has perceptual hashing.

https://github.com/qarmin/czkawka by far the best of anything iv tried

I've used Czkawka (https://github.com/qarmin/czkawka) because it does Lanczos-based image duplicate detection, which makes it more practical for me.

Maybe it's a setting you made or the files, not sure. You can try another software czkawka to see if you get better results with it.

For static images i used https://github.com/qarmin/czkawka and it works well enough. I think. But when i used it on a folder with gifs and their jxl conversions, it shows nothing. SURELY this could not be user error, rrrright?

I used to use DupeGuru which has some photo-specific dupe detection where you can fuzzy match image dupes based on content: https://dupeguru.voltaicideas.net/

But I switched over to czkawka, which has a better interface for comparing files, and seems to be a bit faster: https://github.com/qarmin/czkawka

Unfortunately, neither of these are integrated into Photoprism, so you still have to do some file management outside the database before importing.

I also haven't used Photoprism extensively yet (I think it's running on one of my boxes, but I haven't gotten around to setting it up), but I did find that it wasn't really built for file-based libraries. It's a little more heavyweight, but my research shows that Nextcloud Memories might be a better choice for me (it's not the first-party Nextcloud photos app, but another one put together by the community): https://apps.nextcloud.com/apps/memories

Czkawka can do this.

There's always the well-respected tool, Czkawka. Or, of the CLI is your thing, jdupes is a good option.