Literate programming: Knuth is doing it wrong

This page summarizes the projects mentioned and recommended in the original post on news.ycombinator.com

Our great sponsors
  • WorkOS - The modern identity platform for B2B SaaS
  • InfluxDB - Power Real-Time Data Analytics at Scale
  • SaaSHub - Software Alternatives and Reviews
  • ferret

    Ferret is a free software lisp implementation for real time embedded control systems. (by nakkaya)

  • The whole of ferret's source code is in a single org-mode file, following the literate programming style: https://github.com/nakkaya/ferret/blob/master/ferret.org

  • mu

    Soul of a tiny new machine. More thorough tests → More comprehensible and rewrite-friendly software → More resilient society. (by akkartik)

  • Thanks for all the quotes. This one's my favorite of them, I think. I've chatted with you in the past, and I think we largely see eye to eye. I love LP! I still use it. Here's an x86 VM I maintained for several years: https://github.com/akkartik/mu/tree/main/linux/bootstrap. This (old) essay is merely thoughts on _how_ to do LP. We have some disagreement there, but I hope you'll agree that it's mild in the large scheme of things.

  • WorkOS

    The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.

    WorkOS logo
  • literate-programming

    Creating programs from Markdown code blocks

  • I've been using literate-programming for a decade now to do my side web projects (using a tool I wrote myself, of course, as tradition dictates in the LP space: https://github.com/jostylr/literate-programming ). I find it really useful as a project management tool in dealing with the chaos of several different languages, build tools, etc. With HTML, I've used markdown for text content, pug for more structural setups, plain HTML for header boilerplate, all weaved into the same output document that is just plain HTML. It also is often useful for splitting css between a main site sheet vs local to a file (e.g., a lot of css styling on the main landing page is different than the other content-based pages of a site, at least back when websites had content).

    I've also found it useful when essentially plugging in data or HTML fragments into JavaScript. I can quickly write a trivial dsl that takes in the data in a convenient form and transforms it into a convenient code version. A variant of the transformational technique is when similar code is almost the same, but just needs a little subbing. I can write a single block covering most of the commonality, and then sub in the differences. For example, if writing some arithmetic operator function code, replacing '+' with the other operators.

    An aspect I also love about the approach is having two views of the code. The LP view is more of an outline with blocks subbing in, with some transformations going on. The compiled LP version is one where you can see all the code in full context, minimizing the jumping around at that level, something that cannot be achieved with a bunch of functional calls.

    Over the last few weeks, I have started to learn Elixir. I have noticed that I feel less drawn to use literate-programming for that language. It feels like it is so well designed that literate-programming is almost redundant for it. I am trying to figure out justifying that statement, but it is still early days for me in that language. But it feels like, for instance, the matching on function parameters so that one can avoid if-else if-...-else constructs cuts down on a lot of boilerplate stuff. Admittedly, JavaScript has a lot less of that as well nowadays with all the new language constructs. Maybe it is just about Elixir being a functional language with the safety and little overhead of calling functions that make it more attractive to use functions as the outlining / reordering mechanism. Also, the pipe operator allows steps of transformations to be done easily and clearly, which is super helpful.

    One huge downside of LP is that it allows the dictates of a given language to be worked around, making it harder for others to follow up with the work. Ideally the text of the LP helps with that, but it still is a barrier. This seems less of a problem in the front-end web world because that is just a mess of competing notions, but in something cohesively designed with sensible standards that the community follows (my impression of Elixir), it would become a much bigger downside to strike out on your own path.

  • GNU Emacs

    Mirror of GNU Emacs

  • Context-menus will land in the next Emacs release and are already available on master. Expect more and more functionality to be exposed via them.

    Being emacs, context-menus will also be configurable.

    https://github.com/emacs-mirror/emacs/blob/emacs-28/etc/NEWS...

  • notebook-mode

    GNU Emacs notebook mode

  • There’s some work in this space, such as Nicolas Rougier’s promising notebook-mode[1]. I’m convinced there would be an audience for an OrgBook app that philosophically treated Emacs as an implementation detail. Give it more familiar keybindings, some out of the box nice looking themes, and configure the new context menu functionality as you suggest. Then package it up as something that can be run and installed with or without an existing Emacs.

    It’s hard to imagine experienced Emacsers wanting to lead a project that solves a problem they don’t have, but the community is very friendly so whoever took it on would get plenty of help.

    [1] https://github.com/rougier/notebook-mode

  • itypescript

    ITypescript is a typescript kernel for the Jupyter notebook (A modified version of IJavascript)

  • how about https://github.com/winnekes/itypescript

    (also obserablejs) for a different take on it.

    I have a feeling notebooks are not as popular for javascript since it's quite limiting relative to what a code sandbox can do.

  • lmt

    literate markdown tangle

  • I personally use literate programming to maintain my "dotfiles", mainly NixOS [1], and I _love_ it. I like to describe all possible alternative tools, why I don't use them, possible tools that look nice, random ideas and blog posts that describe parts of my config, add TODOs and screenshots, ... in short everything that is really ugly to do inside source code comments. Also I gain structure; adding headings to a 3000 LOC config is very nice.

    For tangling I use lmt [2], as it works with Markdown and also play nice with Emanote [3] (full syntax highlighting inside the code blocks.). That means all my "dotfiles" are inside my Zettelkasten [4] and can be navigated like any other note I have.

    [1]: https://nixos.org/

    [2]: https://github.com/driusan/lmt

    [3]: https://github.com/srid/emanote

    [4]: https://zettelkasten.de/

  • InfluxDB

    Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

    InfluxDB logo
  • emanote

    Emanate a structured view of your plain-text notes

  • I personally use literate programming to maintain my "dotfiles", mainly NixOS [1], and I _love_ it. I like to describe all possible alternative tools, why I don't use them, possible tools that look nice, random ideas and blog posts that describe parts of my config, add TODOs and screenshots, ... in short everything that is really ugly to do inside source code comments. Also I gain structure; adding headings to a 3000 LOC config is very nice.

    For tangling I use lmt [2], as it works with Markdown and also play nice with Emanote [3] (full syntax highlighting inside the code blocks.). That means all my "dotfiles" are inside my Zettelkasten [4] and can be navigated like any other note I have.

    [1]: https://nixos.org/

    [2]: https://github.com/driusan/lmt

    [3]: https://github.com/srid/emanote

    [4]: https://zettelkasten.de/

  • geom

    2D/3D geometry toolkit for Clojure/Clojurescript

  • This would make sense if Knuth used literate programming primarily for academic papers. But in fact he created WEB for writing TeX and METAFONT, both of which (while their source code was published as a book later) were production systems, and in fact for several decades now he uses CWEB for all programs he writes, including several a week that he writes for himself. (Some of which are online at https://cs.stanford.edu/~knuth/programs.html .) In contrast, apart from the paper he wrote introducing LP, and the two Bentley columns about LP in CACM, I'm not aware of any other academic paper of his that presents programs — at any rate, the total number must be very small.

    The goal is not an "academic paper"; his experience (and that of others who have seriously tried LP) is that it helps with actual writing of programs, less time spent debugging, etc.

    Yes, there are challenges with two or more programmers, but nothing unsurmountable. See "Literate Programming on a Team Project" (https://www.cs.princeton.edu/techreports/1991/302.pdf coauthored by Norman Ramsey, who later developed noweb) and some stories like https://news.ycombinator.com/item?id=17484452 (and https://github.com/thi-ng/geom which went from LP to conventional).

NOTE: The number of mentions on this list indicates mentions on common posts plus user suggested alternatives. Hence, a higher number means a more popular project.

Suggest a related project

Related posts