fclones VS rdfind

Install fclones (fdupes modern equivalent)

I said "fclone" I meant "fclones" (https://github.com/pkolaczk/fclones) -- I missed the "s" at the end

While we're recommending stuff, fclones: https://github.com/pkolaczk/fclones

fclones: Efficient Duplicate File Finder.

Yes, that would be safer when available (although generally files within library dependencies are not modified I think?). It looks like fclones implements this, is faster and is written in Rust https://github.com/pkolaczk/fclones (the last is the most important point of course /s).

I run rdfind[1] as a cronjob to replace duplicates with hardlinks. Works fine!

https://github.com/pauldreik/rdfind

I use rdfind for this.

rdfind is available for macOS; I've been using it on linux: https://github.com/pauldreik/rdfind

You can use rdfind to find all duplicates in your experiments dir and replace files with hardlinks. This way files will occupy disk space only once and all inode references will be to the same disk location.

I use rdfind to deal with this: https://github.com/pauldreik/rdfind

Yeah, I periodically download the full drives and just deduplicate with rdfind hardlinking identical files.