dedupe VS fd

I wrote https://github.com/Gumnos/dedupe which sounds like it might be useful to you. It's faster than several of the alternatives I've found (many run the checksum across the whole of every file, this uses the file-size as a first-line discriminator, and only if the files are the same size does it go to the trouble of checking the checksum of the files). I designed it for creating hard-links in my media collection, but in the --dry-run mode, it should emit the file-names allowing you to pass it to xargs to remove them if it looks copacetic.

You can use my dedupe.py script with the dry-run flag (-n) to find all the duplicates on your drive. If you run it without the dry-run flag, it will attempt to make hard-links so that each file exists only once on the drive with multiple hard-links to the underlying file. It should be pretty fast, only needing to checksum file-content in the event that files have the same size (several other such deduplication methods work by checksumming every file on the drive which can be slow).

a file-deduplication utility that hard-links duplicate files to save space (our family photo gallery gets pics put in multiple albums for various audiences, so I can cut down on a lot of duplication with this)

ripgrep: A super-fast file searcher. You can install it using your system's package manager (e.g., brew install ripgrep on macOS). fd: Another blazing-fast file finder. Installation instructions can be found here: https://github.com/sharkdp/fd

hyperfine is such a great tool that it's one of the first I reach for when doing any sort of benchmarking.

I encourage anyone who's tried hyperfine and enjoyed it to also look at sharkdp's other utilities, they're all amazing in their own right with fd[1] being the one that perhaps get the most daily use for me and has totally replaced my use of find(1).

[1]: https://github.com/sharkdp/fd

You call it with `n` and get an interactive fuzzy search for your directories. If you do `n ` instead, it’ll start the find with `` already filled in (and if there’s only one match, jump to it directly). The `ls` is optional but I find that I like having the contents visible as soon as I change a directory.

I’m also including iCloud Drive but excluding the Library directory as that is too noisy. I have a separate `nl` function which searches just inside `~/Library` for when I need it, as well as other specialised `n` functions that search inside specific places that I need a lot.

¹ https://github.com/sharkdp/fd

² https://github.com/junegunn/fzf

Many (most?) of them have been overhauled with success. For find there is fd[1]. There's batcat, exa (ls), ripgrep, fzf, atuin (history), delta (diff) and many more.

Most are both backwards compatible and fresh and friendly. Your hardwon muscle memory still of good use. But there's sane flags and defaults too. It's faster, more colorful (if you wish), better integration with another (e.g. exa/eza or aware of git modifications). And, in my case, often features I never knew I needed (atuin sync!, ripgrep using gitignore).

1 https://github.com/sharkdp/fd

Descubra mais sobre o fd em: https://github.com/sharkdp/fd

AFAIK there is a find replacement with sane defaults: https://github.com/sharkdp/fd , a lot of people I know love it.

However, I already have this in my muscle memory:

fd

> without zsh globs I have to remember find syntax

My "solution" to this is using https://github.com/sharkdp/fd (even when in zsh and having glob support). I'm not sure if using a tool that's not present by default would be suitable for your use cases, but if you're considering alternate shells, I suspect you might be

Nice to see other alternatives to find. I personally use fd (https://github.com/sharkdp/fd) a lot, as I find the UX much better. There is one thing that I think could be better, around the difference between "wanting to list all files that follow a certain pattern" and "wanting to find one or a few specific files". Technically, those are the same, but an issue I'll often run into is wanting to search something in dotfiles (for example the Go tools), use the unrestricted mode, and it'll find the few files I'm looking for, alongside hundreds of files coming from some cache/backup directory somewhere. This happens even more with rg, as it'll look through the files contents.

I'm not sure if this is me not using the tool how I should, me not using Linux how I should, me using the wrong tool for this job, something missing from the tool or something else entirely. I wonder if other people have this similar "double usage issue", and I'm interested in ways to avoid it.