documentation-framework VS MarginaliaSearch

Writing Great Documentation

I forget the terminology, but there's a good "grid" breakdown of documentation types (I think this one: https://documentation.divio.com ) that I've simplified a bit for the internal documentation I'm involved with.

* README, HOWTO, INFO, PROJECT, DESIGN, NOTES, FAQ

When I pull down a `git` repo, I read the `README.md` (of course). I make my own `NOTES.md` (eg: `.gitignore`'d) of what commands, environment variables, useful blog posts, search results, whatever. Rarely do I share or encourage sharing of `NOTES.md` wholesale, but it's helpful to be able to pull out a few snippets or re-orient myself when coming back to that software/project.

Then, other documents get prefixed with "HOWTO-Do-Some-Specific-Thing.md", or "INFO-Some-Particular-Component.md".

"PROJECT-...", and "DESIGN-..." are "dangerous" ones in that they can quickly fall out of date, but they can be very useful while they're being actively managed. I guess personally I've started making sure to include dates or "eras" in the title, eg: "PROJECT-[2024-Feb]-Add-Foo-Support.md" or "DESIGN-[2024-02-14]-...". Stuff that's outlived its usefulness can probably be moved to an `ARCHIVE/...` in case you need it later, but keep it out of the way from confusing newcomers 1-3 years from now.

"FAQ-..." almost never comes into play (hopefully) b/c it should mostly get absorbed into "HOWTO-..." or product improvements, and few products seem to rise to the level of needing FREQUENTLY asked questions. Ideally FAQ's would "go away" with work on the product or other documentation, but I've had some success with it as like sales-oriented (and ideally: sales-managed) FAQ / Canned Customer Response learnings.

Putting it all together you get something like:

  * README.md

Resource: Documentation Best Practices - GitBook

I would not suggest people to follow this in 2024 if they are building any system of non-trivial scope and expect it to be adopted by others who are not required to adopt it.

Back in 2017, I compared "code as documentation" to being dropped into on the street of an unfamiliar city, while a good documentation can serve as a map of the city. [1]

Nearly all recent successful efforts for large new systems understand the value of both high-level overviews and detailed examples / onboarding materials to make adoption easier. When solutions to a certain problem are abundant, people do not need to settle for options that do not have great supporting documentation of the four primary kinds. [2]

[1] https://speakerdeck.com/maxvt/i-got-a-lot-of-problems-with-i...

[2] https://documentation.divio.com/

In the whole spectrum of documentation, man pages were designed to cater for a very specific need: information-oriented reference data. Check https://documentation.divio.com/ for a wonderful classification of documentation into four quadrants: reference, explanations, tutorials, and how-to guides. On the other hand, one can write info files for any of the use cases. That does not make info format inherently better or worse than man pages.

During the years, there have been many attempts to bridge the format gap, and convert texts from one representation to another. One of the most ambitious ones was in Tkman, a man viewer built on then Tcl/Tk system. Its really interesting part was the inclusion of rman, or RosettaMan, a converter of text to a somewhat abstract representation that could then be viewed via a GUI.

I personally look for well-crafted man pages as a sign of quality in software and try to provide them in everything I develop. I admit that I don't often find the time or motivation to write non-reference documentation (like tutorials).

I ran into the divio documentation guide recently that seems to have some awesome "how to write docs" docs

https://documentation.divio.com/ is a good overview of the "four types of documentation" paradigm: tutorials, how-to guides, explanations, and reference have to all exist.

One of my major gripes with the JS/TS ecosystem is that "explanations" are sorely lacking. See https://www.typescriptlang.org/tsconfig for the relevant documentation for tsconfig files. Tutorials are on the page, how-to guides abound on the wider internet (like the OP), and the linked TSConfig Reference and JSON Schema (used in code completion in IDEs) are together absolutely massive.

But an explanation is missing! There is no official documentation about how different options interact to say: as I'm walking a file tree as the Typescript compiler, this is how I will interpret a certain file I encounter, what will be outputted, and how that will be interpreted by bundlers and browsers, especially in an ESM world.

https://medium.com/extra-credit-by-guild/tsconfig-json-demys... is in the right direction, but outdated as ESM has become much more popular in the past 3 years, and still organized by option (so it's already somewhat in the "reference" world).

IMO even independent of documentation, the industry's move to ESM is problematic: https://gist.github.com/joepie91/bca2fda868c1e8b2c2caf76af7d... describes many of the issues. But they're certainly exacerbated by good explanation-style documentation that helps people understand how ESM works under the hood!

I really like the system detailed here: https://documentation.divio.com/. That's targeted more towards externally visible docs, but IMO adapts pretty well as for internal resources too.

Well it's open source and contributions are welcome, though it's a fairly sprawling java project that's probably not the easiest to get into still (through no lack of effort making it more accessible): https://github.com/MarginaliaSearch/MarginaliaSearch

If you have spare dollars but not time, you can also contribute to the war chest: https://about.marginalia-search.com/article/supporting/

Based solely upon the title and the first commit's date, I'm guessing it's this: https://github.com/MarginaliaSearch/MarginaliaSearch/pull/99

Marginalia Search is an AI-free[1] search engine, mostly because I don't think search is much helped by AI. The sorts of problems that does better than traditional IR-methods based search are things traditional search engines never did particularly well anyway.

https://search.marginalia.nu/

Have you tried https://search.marginalia.nu yet?

If there's no default search deal, will there be a option to pick which search engine you use? Or will it just default to something like using https://search.marginalia.nu/ out of the box?

I'll add some more "small web" links, for those interested. Thanks for yours!

* https://search.marginalia.nu/

* https://wiby.me/surprise/

* https://neustadt.fr/essays/the-small-web/

* https://www.reddit.com/r/AskReddit/comments/3hn0a7/what_smal...

> Then maybe you might be interested in Marginalia Search https://search.marginalia.nu/

I have tried it before, but it appears to me that it dismisses all commercial content altogether, which is not what I want. That is throwing out the baby with the bathwater.

For example, I searched for "The Vietnam of computer science" (Without the quotes) and it returned zero results.

> I think a larger concern is how you'll address the Bus Factor going forward

I can't speak to how much energy it is to go from code to serving requests, but FWIW the code is AGPLv3 and seems to be updated regularly https://github.com/MarginaliaSearch/MarginaliaSearch/blob/v2...

Regarding the last sentence: The problem is that capitalism knows no limits. Sure, it would be nice to pay a monthly subscription for genuinely good and desirable content/search results...

But what if the CEO of the service provider needs another $5m bonus? What if the stock needs to go up so that the shareholder gamblers can get more dividend paid? What if all of a sudden the service gets bought out?

The truth is that what you are seeking is more likely to come from someone who is just passionate about it with not that much motivation based on profit. That doesn't mean that this entity or person can't be financially supported but it gets problematic when profit is the _main_ incentive.

For a good example of an interesting search engine built by a single guy, see Marginalia: https://search.marginalia.nu/

Have you heard of https://www.marginalia.nu/ in general, and especially the https://search.marginalia.nu/ from there?