Duplicates

Open-source projects categorized as Duplicates
Edit details
Language: + Rust + TypeScript + C + C++ + JavaScript + Python + Go
Topics: duplicate-files duplicate-detection Rust Deduplication Filesystem
SaaSHub - Software Alternatives and Reviews
SaaSHub helps you find the best software and product alternatives
www.saashub.com
Sponsored

Top 13 Duplicate Open-Source Projects

  • czkawka

    Multi functional app to find duplicates, empty folders, similar images etc.

    Project mention: Is there software to compress large but similar files? | /r/DataHoarder | 2023-12-11

  • jscpd

    Copy/paste detector for programming source code.

    Project mention: Copy/paste detector for programming source code, supports 150 formats | news.ycombinator.com | 2024-01-29

  • InfluxDB

    Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

  • rmlint

    Extremely fast tool to remove duplicates and other lint from your filesystem

    Project mention: fdupes: Identify or Delete Duplicate Files | news.ycombinator.com | 2023-11-02

    My preferred solution is rmlint [https://github.com/sahib/rmlint] mostly because it also looks at duplicate directories. It produces a bash script instead of deleting anything itself, so you can examine it before running the script it made.

  • yarn-deduplicate

    Deduplication tool for yarn.lock files

  • dupd

    CLI utility to find duplicate files

  • backdown

    A deduplicator

  • removedupes

    Remove Duplicate Messages

  • WorkOS

    The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.

  • cbird

    Command-line program for managing a media collection, with focus on Content-Based Image Retrieval (Computer Vision) methods for finding duplicates.

    Project mention: Similar photo finder for screenshots | /r/DataHoarder | 2023-07-04

    Try cbird (my program). In addition to playing with thresholds, there are a couple of options that might help. Also if you can share the screenshots I'd be interested in testing.

  • Katalog

    Katalog is an application to manage catalogs of disks and files to search and get statistics.

  • dude

    Duplicates Detector is a cross-platform GUI utility for finding duplicate files, allowing you to delete or link them to save space. Duplicate files are displayed and processed on two synchronized panels for efficient and convenient operation. (by PJDude)

    Project mention: fdupes: Identify or Delete Duplicate Files | news.ycombinator.com | 2023-11-02

    Hi. I recommend my little program, the bottleneck is the gui in tkinter, but maybe it will be useful to someone:

    https://github.com/PJDude/dude

  • akin

    Rust crate for writing repetitive code easier and faster.

  • duplikaatti

    Remove duplicate files.

  • samanlainen

    Delete duplicate files

NOTE: The open source projects on this list are ordered by number of github stars. The number of mentions indicates repo mentiontions in the last 12 Months or since we started tracking (Dec 2020). The latest post mention was on 2024-01-29.

Duplicates related posts

Index

What are some of the best open-source Duplicate projects? This list will help you:

Project Stars
1 czkawka 17,336
2 jscpd 4,491
3 rmlint 1,768
4 yarn-deduplicate 1,372
5 dupd 109
6 backdown 95
7 removedupes 78
8 cbird 71
9 Katalog 55
10 dude 52
11 akin 42
12 duplikaatti 15
13 samanlainen 6

Sponsored
SaaSHub - Software Alternatives and Reviews
SaaSHub helps you find the best software and product alternatives
www.saashub.com