C Deduplication

Open-source C projects categorized as Deduplication

Top 6 C Deduplication Projects

  • libpostal

    A C library for parsing/normalizing street addresses around the world. Powered by statistical NLP and open geo data.

  • rmlint

    Extremely fast tool to remove duplicates and other lint from your filesystem

  • Project mention: fdupes: Identify or Delete Duplicate Files | news.ycombinator.com | 2023-11-02

    My preferred solution is rmlint [https://github.com/sahib/rmlint] mostly because it also looks at duplicate directories. It produces a bash script instead of deleting anything itself, so you can examine it before running the script it made.

  • InfluxDB

    Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

    InfluxDB logo
  • kvdo

    A kernel module which provide a pool of deduplicated and/or compressed block storage.

  • vdo

    Userspace tools for managing VDO volumes.

  • dupd

    CLI utility to find duplicate files

  • swuniq

    A command-line tool for deduplicating entries in a file or stream with constant memory usage

NOTE: The open source projects on this list are ordered by number of github stars. The number of mentions indicates repo mentiontions in the last 12 Months or since we started tracking (Dec 2020).

C Deduplication related posts

  • Ask HN: Open-source Windows 11 backup solutions

    4 projects | news.ycombinator.com | 4 Apr 2024
  • File Servers... how are you handling duplicates

    1 project | /r/sysadmin | 8 Dec 2023
  • fdupes: Identify or Delete Duplicate Files

    13 projects | news.ycombinator.com | 2 Nov 2023
  • Johnny Decimal

    4 projects | news.ycombinator.com | 13 Jun 2023
  • Jdupes: A powerful duplicate file finder

    1 project | news.ycombinator.com | 6 Jun 2023
  • Does jdupes do a 'dry run' if you just specify directory(s) and no other options

    1 project | /r/linuxquestions | 4 Jun 2023
  • replace duplicates with hard links - I think jdupes is the answer, or maybe fclones (I have questions)

    1 project | /r/linuxquestions | 4 Jun 2023
  • A note from our sponsor - InfluxDB
    www.influxdata.com | 8 May 2024
    Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality. Learn more →

Index

What are some of the best open-source Deduplication projects in C? This list will help you:

Project Stars
1 libpostal 3,953
2 rmlint 1,778
3 kvdo 237
4 vdo 189
5 dupd 109
6 swuniq 5

Sponsored
SaaSHub - Software Alternatives and Reviews
SaaSHub helps you find the best software and product alternatives
www.saashub.com