geom VS literate-programming

On the flip side, I got to read some of the Clojure source code, which was very educational. I also got to understand a bit more the usefulness of protocols (using defprotocol and defrecord to provide several implementations). Here it was very useful to read the source code of thi-ng/geom.

This would make sense if Knuth used literate programming primarily for academic papers. But in fact he created WEB for writing TeX and METAFONT, both of which (while their source code was published as a book later) were production systems, and in fact for several decades now he uses CWEB for all programs he writes, including several a week that he writes for himself. (Some of which are online at https://cs.stanford.edu/~knuth/programs.html .) In contrast, apart from the paper he wrote introducing LP, and the two Bentley columns about LP in CACM, I'm not aware of any other academic paper of his that presents programs — at any rate, the total number must be very small.

The goal is not an "academic paper"; his experience (and that of others who have seriously tried LP) is that it helps with actual writing of programs, less time spent debugging, etc.

Yes, there are challenges with two or more programmers, but nothing unsurmountable. See "Literate Programming on a Team Project" (https://www.cs.princeton.edu/techreports/1991/302.pdf coauthored by Norman Ramsey, who later developed noweb) and some stories like https://news.ycombinator.com/item?id=17484452 (and https://github.com/thi-ng/geom which went from LP to conventional).

I've been using literate-programming for a decade now to do my side web projects (using a tool I wrote myself, of course, as tradition dictates in the LP space: https://github.com/jostylr/literate-programming ). I find it really useful as a project management tool in dealing with the chaos of several different languages, build tools, etc. With HTML, I've used markdown for text content, pug for more structural setups, plain HTML for header boilerplate, all weaved into the same output document that is just plain HTML. It also is often useful for splitting css between a main site sheet vs local to a file (e.g., a lot of css styling on the main landing page is different than the other content-based pages of a site, at least back when websites had content).

I've also found it useful when essentially plugging in data or HTML fragments into JavaScript. I can quickly write a trivial dsl that takes in the data in a convenient form and transforms it into a convenient code version. A variant of the transformational technique is when similar code is almost the same, but just needs a little subbing. I can write a single block covering most of the commonality, and then sub in the differences. For example, if writing some arithmetic operator function code, replacing '+' with the other operators.

An aspect I also love about the approach is having two views of the code. The LP view is more of an outline with blocks subbing in, with some transformations going on. The compiled LP version is one where you can see all the code in full context, minimizing the jumping around at that level, something that cannot be achieved with a bunch of functional calls.

Over the last few weeks, I have started to learn Elixir. I have noticed that I feel less drawn to use literate-programming for that language. It feels like it is so well designed that literate-programming is almost redundant for it. I am trying to figure out justifying that statement, but it is still early days for me in that language. But it feels like, for instance, the matching on function parameters so that one can avoid if-else if-...-else constructs cuts down on a lot of boilerplate stuff. Admittedly, JavaScript has a lot less of that as well nowadays with all the new language constructs. Maybe it is just about Elixir being a functional language with the safety and little overhead of calling functions that make it more attractive to use functions as the outlining / reordering mechanism. Also, the pipe operator allows steps of transformations to be done easily and clearly, which is super helpful.

One huge downside of LP is that it allows the dictates of a given language to be worked around, making it harder for others to follow up with the work. Ideally the text of the LP helps with that, but it still is a barrier. This seems less of a problem in the front-end web world because that is just a mess of competing notions, but in something cohesively designed with sensible standards that the community follows (my impression of Elixir), it would become a much bigger downside to strike out on your own path.

A system that I created that does all of that is https://github.com/jostylr/literate-programming It does work and I tweak every now and then, but it certainly is not in an ideal state. It is a npm installable command line program and there is a 10% written book linked to from the README (free to read on Leanpub in HTML style). Almost every command is documented in there, but just barely.

It is a markdown-based tool. I had experimented with creating a different tool called pieceful-programming which would allow for changing the basic literate style (say instead of markdown, using asciidoc). But I did not complete it (yet?).

I used my tool for web programming, allowing me to freely arrange backend and frontend code as I see fit.