Ask HN: Best tool for de-duplication of local files

• periscope

  1 370 4.7 Go

  Periscope gives you "duplicate vision" to help you organize and de-duplicate your files without losing data 🔭 (by anishathalye)

I haven't tried this out personally but have used some of his other tools: https://github.com/anishathalye/periscope

• my-photo-timeline

  1 41 0.0 Scala

  A simple command-line app to organize your local photos by dates (year/month) on your local file system, for those who don't store photos in the cloud

  

I had a similar issue with photos/videos and ended up building a cli app to organize everything, it has worked for my use case relatively well, still, there are uncovered corner cases. For example, this compares hashes only instead of the same photo in a different dimension: https://github.com/wiringbits/my-photo-timeline

I understand that https://photostructure.com/ has a far more sophisticated dedup algorithm, which can be worth a try.

• InfluxDB

  www.influxdata.com featured

  Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

  

• czkawka

  361 17,501 7.7 Rust

  Multi functional app to find duplicates, empty folders, similar images etc.

Perhaps [qarmin/czkawka: Multi functional app to find duplicates, empty folders, similar images etc.](https://github.com/qarmin/czkawka) and it's alternatives mentioned in the readme?

Suggest a related project