nimsearch VS home-ops

You are welcome. Thanks are too rarely offered. :-)

You may also be interested in word stemming ( such as used by snowball stemmer in https://github.com/c-blake/nimsearch ) or other NLP techniques, but I don't know how internationalized/multi-lingual that stuff is, but conceptually you might want "series of stemmed words" to be the content fragments of interest.

Similarity scores have many applications. Weights on graph of cancelled downloads ranked by size might be one. :)

Of course, for your specific "truncation" problem, you might also be able to just do an edit distance against the much smaller filenames and compare data prefixes in files or use a SHA256 of a content-based first slice. ( There are edit distance algos in Nim in https://github.com/c-blake/cligen/blob/master/cligen/textUt.... as well as in https://github.com/c-blake/suggest ).

Or, you could do a little program like ndup/sh/ndup to create a "mirrored file tree" of such content-based slices then you could use any true duplicate-file finder (like https://github.com/c-blake/bu/blob/main/dups.nim) on the little signature system to identify duplicates and go from path suffixes in those clusters back to the main filesystem. Of course, a single KV store within one or two files would be more efficient than thousands of tiny files. There are many possibilities.

It's all in the Nim programming language, but if you prefer reading code or running diffs then you might get a vague sense of (some) low level nuts & bolts from: https://github.com/c-blake/nimsearch

These are great operational wins. Agreed very much that having autonomic (can fix itself) systems at your back is a massive game changer. De-crustifies the act of running things.

The other win is that there's a substantial cultural base to this way to go. Folks have been doing selfhosting for ages, but everyone has their own boutique setup some their way. A couple tools and techniques could be shared, but mostly everyone took blank slate configs & built their own system up, & added their own monitoring & operational scripts.

https://github.com/onedr0p/home-ops is a set of helm scripts and other tools that is widely widely used, and there's a lot more like it. It's a huge build out, using convention and a common platform to enable portable knowledge & sharing.

Self hosting did not have intellectual scale out at it's back, before Kubernetes came along. Docker and ansible and others have been around, but theres never been remotely the success there has been today in empowering users to setup & run complex services.

We really have clawed out of the server-hugging jungle &started building some villages. It's wonderful to see.

Going thru this repo https://github.com/onedr0p/home-ops

Take a look at my open source GitOps repo managed by Flux here: https://github.com/onedr0p/home-ops

Im fully onboard with the geneneral idea as a target.

Right now it's for early early adopters. Hosting stuff is still a painm But we are getting better at hosting stuff, finding stable patterns, paving the path. Hint, it's not doing less, it's not simpler options: it's adopting & making our own industrial scale tooling. https://github.com/onedr0p/home-ops is a great early & still strong demonstration; the up front cost od learning is high, but there's the biggest ecosystem of support you can imagine, and once you recognize the patterns, you can get into flow states, make stuff happen, with extreme leverage far beyond where humanity has ever been. Building the empowered individual is happening, and we're using stable good patterns that will mean the individual isnt so off on their own doing ops- they'll have a lot more accrued human experiene at their back, their running of services isnt as simple to understand from the start but goes much much further, is much more mature & well supported in the long run.

My Kubernetes cluster, deployments, infrastructure provisioning is all available over here on Github.

For example: https://github.com/onedr0p/home-ops/pull/4528

I'm running minio in my cluster with NFS backend just fine. You can see my deployment of it here.