Backing Up Data: Tips/Advice for Tons of Unorganized Data and Duplicate Files from Multiple Sources

This page summarizes the projects mentioned and recommended in the original post on /r/DataHoarder
Deduplication Python C duplicate-files Fast

InfluxDB - Power Real-Time Data Analytics at Scale
Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
www.influxdata.com
featured
SaaSHub - Software Alternatives and Reviews
SaaSHub helps you find the best software and product alternatives
www.saashub.com
featured

  • dupeguru

    Find duplicate files

  • fdupes

    FDUPES is a program for identifying or deleting duplicate files residing within specified directories.

  • InfluxDB

    Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

    InfluxDB logo

  • jdupes

    Discontinued A powerful duplicate file finder and an enhanced fork of 'fdupes'.

  • rdfind

    find duplicate files utility

NOTE: The number of mentions on this list indicates mentions on common posts plus user suggested alternatives. Hence, a higher number means a more popular project.

Suggest a related project

Related posts

  • Tools to find duplicate files on muliple file servers

    2 projects | /r/sysadmin | 4 Oct 2022

  • Recommendation Needed. I just received several terabytes of unstructured data. It appears that the user would reate a backup of his files "Backup 1", "Backup 2", etc, and then drag entire copies of his files over. There are thousands of duplicate files. Can someone recommend a good file dedupe app?

    2 projects | /r/sysadmin | 2 Sep 2022

  • Suggestions on how to identify & report on old stale data in file shares?

    4 projects | /r/sysadmin | 10 May 2022

  • Photo Cluster F* on my hard disk - can Synology Photos help me out?

    2 projects | /r/synology | 17 Jul 2021

  • File Servers... how are you handling duplicates

    1 project | /r/sysadmin | 8 Dec 2023