MarginaliaSearch VS documentation-framework

> I think a larger concern is how you'll address the Bus Factor going forward

I can't speak to how much energy it is to go from code to serving requests, but FWIW the code is AGPLv3 and seems to be updated regularly https://github.com/MarginaliaSearch/MarginaliaSearch/blob/v2...

Regarding the last sentence: The problem is that capitalism knows no limits. Sure, it would be nice to pay a monthly subscription for genuinely good and desirable content/search results...

But what if the CEO of the service provider needs another $5m bonus? What if the stock needs to go up so that the shareholder gamblers can get more dividend paid? What if all of a sudden the service gets bought out?

The truth is that what you are seeking is more likely to come from someone who is just passionate about it with not that much motivation based on profit. That doesn't mean that this entity or person can't be financially supported but it gets problematic when profit is the _main_ incentive.

For a good example of an interesting search engine built by a single guy, see Marginalia: https://search.marginalia.nu/

Have you heard of https://www.marginalia.nu/ in general, and especially the https://search.marginalia.nu/ from there?

There's a decent amount of people still keeping the fire burning for the "old web." It takes a little digging, but it's out there.

Some links for you:

* https://wiby.me/ — search engine that emphasizes simple/plain/hobbyist pages. Try the "surprise me" link a few times.

* https://neustadt.fr/essays/the-small-web/ — article, "Rediscovering the Small Web"

* https://search.marginalia.nu/ — author (hangs out on HN sometimes, too (marginalia_nu)

Actually, here's a link to a similar discussion on an old HN thread: https://news.ycombinator.com/item?id=30783391

I built a recipe detector. You can, you know, train some sort of AI model to do this like with fasttext, or maybe do naive bayesian inference, but as it turns out, you can also:

https://github.com/MarginaliaSearch/MarginaliaSearch/blob/ma...

It works annoyingly well.

For some values of "nobody"; this westerner enjoys https://search.marginalia.nu (in addition to more common engines) and has high hopes for the new site browser:

Result ranking takes a lot of variables, and factors like excessive tracking and affiliate links is one of them in my search engine.

You can poke around in the result valuation code here: https://github.com/MarginaliaSearch/MarginaliaSearch/blob/ma...

#1 cause in the decline of Google maybe. https://search.marginalia.nu/ seems to manage though, so maybe Google just doesn't care.

Congratulations on shipping.

I see a lot of focus on startups, AI tools, productivity hacks, tech stacks, etc. What audience do you have in mind? I personally find that the most interesting sites on HN are outside of your scope here (examples: https://ciechanow.ski/, https://neal.fun/, https://search.marginalia.nu/).

I forget the terminology, but there's a good "grid" breakdown of documentation types (I think this one: https://documentation.divio.com ) that I've simplified a bit for the internal documentation I'm involved with.

* README, HOWTO, INFO, PROJECT, DESIGN, NOTES, FAQ

When I pull down a `git` repo, I read the `README.md` (of course). I make my own `NOTES.md` (eg: `.gitignore`'d) of what commands, environment variables, useful blog posts, search results, whatever. Rarely do I share or encourage sharing of `NOTES.md` wholesale, but it's helpful to be able to pull out a few snippets or re-orient myself when coming back to that software/project.

Then, other documents get prefixed with "HOWTO-Do-Some-Specific-Thing.md", or "INFO-Some-Particular-Component.md".

"PROJECT-...", and "DESIGN-..." are "dangerous" ones in that they can quickly fall out of date, but they can be very useful while they're being actively managed. I guess personally I've started making sure to include dates or "eras" in the title, eg: "PROJECT-[2024-Feb]-Add-Foo-Support.md" or "DESIGN-[2024-02-14]-...". Stuff that's outlived its usefulness can probably be moved to an `ARCHIVE/...` in case you need it later, but keep it out of the way from confusing newcomers 1-3 years from now.

"FAQ-..." almost never comes into play (hopefully) b/c it should mostly get absorbed into "HOWTO-..." or product improvements, and few products seem to rise to the level of needing FREQUENTLY asked questions. Ideally FAQ's would "go away" with work on the product or other documentation, but I've had some success with it as like sales-oriented (and ideally: sales-managed) FAQ / Canned Customer Response learnings.

Putting it all together you get something like:

  * README.md

Resource: Documentation Best Practices - GitBook

I would not suggest people to follow this in 2024 if they are building any system of non-trivial scope and expect it to be adopted by others who are not required to adopt it.

Back in 2017, I compared "code as documentation" to being dropped into on the street of an unfamiliar city, while a good documentation can serve as a map of the city. [1]

Nearly all recent successful efforts for large new systems understand the value of both high-level overviews and detailed examples / onboarding materials to make adoption easier. When solutions to a certain problem are abundant, people do not need to settle for options that do not have great supporting documentation of the four primary kinds. [2]

[1] https://speakerdeck.com/maxvt/i-got-a-lot-of-problems-with-i...

[2] https://documentation.divio.com/

In the whole spectrum of documentation, man pages were designed to cater for a very specific need: information-oriented reference data. Check https://documentation.divio.com/ for a wonderful classification of documentation into four quadrants: reference, explanations, tutorials, and how-to guides. On the other hand, one can write info files for any of the use cases. That does not make info format inherently better or worse than man pages.

During the years, there have been many attempts to bridge the format gap, and convert texts from one representation to another. One of the most ambitious ones was in Tkman, a man viewer built on then Tcl/Tk system. Its really interesting part was the inclusion of rman, or RosettaMan, a converter of text to a somewhat abstract representation that could then be viewed via a GUI.

I personally look for well-crafted man pages as a sign of quality in software and try to provide them in everything I develop. I admit that I don't often find the time or motivation to write non-reference documentation (like tutorials).

I ran into the divio documentation guide recently that seems to have some awesome "how to write docs" docs

https://documentation.divio.com/ is a good overview of the "four types of documentation" paradigm: tutorials, how-to guides, explanations, and reference have to all exist.

One of my major gripes with the JS/TS ecosystem is that "explanations" are sorely lacking. See https://www.typescriptlang.org/tsconfig for the relevant documentation for tsconfig files. Tutorials are on the page, how-to guides abound on the wider internet (like the OP), and the linked TSConfig Reference and JSON Schema (used in code completion in IDEs) are together absolutely massive.

But an explanation is missing! There is no official documentation about how different options interact to say: as I'm walking a file tree as the Typescript compiler, this is how I will interpret a certain file I encounter, what will be outputted, and how that will be interpreted by bundlers and browsers, especially in an ESM world.

https://medium.com/extra-credit-by-guild/tsconfig-json-demys... is in the right direction, but outdated as ESM has become much more popular in the past 3 years, and still organized by option (so it's already somewhat in the "reference" world).

IMO even independent of documentation, the industry's move to ESM is problematic: https://gist.github.com/joepie91/bca2fda868c1e8b2c2caf76af7d... describes many of the issues. But they're certainly exacerbated by good explanation-style documentation that helps people understand how ESM works under the hood!

I really like the system detailed here: https://documentation.divio.com/. That's targeted more towards externally visible docs, but IMO adapts pretty well as for internal resources too.