rsync VS rmlint

So we have our SSH connection all oiled and ready to go. In the next post on the series, we will see how can we use Rsync to sync our music files, bidirectionally. See ya, on the next tab.

Rsync is a command-line tool that enables syncing files and directories between the local and remote system. It's highly flexible and it's similar to scp, if you worked on scp before, the rsync is just scp on steroids.

rsync can be used to synchronize a local disk to the pCloud drive p. Works similarly as the Sync option of the pCloud Drive app. May be useful if one prefers a bulk upload once a day over a continuous synchronization.

Tools that can be used to handle this include Rsync, Duplicati, Cohesity

To overcome this issue, you can use rsync, a powerful command-line tool for synchronizing files and directories, along with a bash script that excludes the node_modules folder and also filter out anything in a .gitignore file that you specify. In this article, I'll guide you through the process of setting up and using this bash script to sync your Node.js project while ignoring the node_modules folder.

On OL9 I installed rsync using https://github.com/WayneD/rsync/blob/master/INSTALL.md and installing dependencies using the For Fedora 33 commands where possible except xxhash which wasn't found, so I used https://oraclelinux.pkgs.org/9/ol9-codeready-builder-aarch64/xxhash-devel-0.8.1-3.el9.aarch64.rpm.html

rsync will probably work better for this use case

I would prefer to use a revision control system, like git if keeping multiple versions of documents is important. Otherwise, if you don't find anything else to your liking, you could write your own scripts based on rsync with the clone-dest, detect-renamed, and detect-renamed-lax patches.

Give rsync a try, it should handle UTF filenames just fine.

My preferred solution is rmlint [https://github.com/sahib/rmlint] mostly because it also looks at duplicate directories. It produces a bash script instead of deleting anything itself, so you can examine it before running the script it made.

After I removed duplicates (with help of https://github.com/sahib/rmlint ), I migrated my photos to an ordinary zpool instead.

For the technically savvy among you there is an excellent open source program called ‘rmlint’ (aka. Remove Lint). It is excellent at finding duplicates and saved me terabytes of space.

You don’t say if you are on Windows or Unix. I have used rmlint successfully in the past.

dupeguru or rmlint

rmlint, my friend, is the last tool you will ever need for this

I use and test assorted duplicate finders regularly.

fdupes is the classic (going way way back) but it's really very slow, not worth using anymore.

The four I know are worth trying these days (depending on data set, hardware, file arrangement and other factors, any one of these might be fastest for a specific use case) are https://github.com/jbruchon/jdupes , https://github.com/pauldreik/rdfind , https://github.com/jvirkki/dupd , https://github.com/sahib/rmlint

Had not encountered fclones before, will give it a try.