KeenType VS keenwrite-themes

> modern languages like Markdown

Markdown was created in 2004. From the creator:

> ... the single biggest source of inspiration for Markdown’s syntax is the format of plain text email.

Email goes back to 1965, though I suspect Markdown's influence stems from the more widely adopted email usage from the 1990s.

> Part of LaTeX's success was the absolutely beautiful documents it can make with nothing but a personal computer.

I'd say that was TeX's success, with LaTeX bolted on later to greatly improve TeX's extensibility. Keep in mind, there are a number of TeX-centric implementations beyond LaTeX. For example, my fork of NTS, called KeenType, is a pure Java version of TeX that can typeset beautifully and has at its core Knuth's original TeX files.

https://github.com/DaveJarvis/KeenType/tree/main/tex/src/mai...

My port of the New Typesetting System (NTS), called KeenType, whittles down the Java-based implementation to a few of Knuth's files. Namely, "plain.tex" and "hyphen.tex":

https://github.com/DaveJarvis/KeenType/tree/main/tex/src/mai...

Getting familiar with the fonts required understanding the difference between font metrics (TFM files) and the fonts themselves. To make matters a little less straightforward, Knuth created a special character mapping for indexes into the fonts. It was not easy to find a font that mapped those glyphs exactly. The closest font was BaKoMa:

https://github.com/DaveJarvis/KeenType/tree/main/tex/src/mai...

This required hard-coding a mapping between Knuth's code points and the actual code points in the target font:

https://github.com/DaveJarvis/KeenType/blob/989dbe26f68eda75...

A while ago I optimized KeenType, which although not JavaScript-based, generates SVG images, which can be embedded into web pages.

https://github.com/DaveJarvis/keentype

The following tutorial shows the real-time rendering speed of KeenType within my text editor, KeenWrite:

https://youtu.be/vgyYXwwF_lc?list=PLB-WIt1cZYLm1MMx2FBG9KWzP...

In Java, many number-to-string implementations use NumberFormat. This is abysmally slow if the problem domain doesn't require internationalization, which is the case for machine-readable file formats, such as SVG. When I performance tested JMathTeX for rendering TeX, the bottleneck for converting TeX into SVG elements was JFreeSVG's use of NumberFormat[0]. Replacing NumberFormat with RyuDouble doubled the throughput[1]. (Reusing a StringBuilder for to concatenate strings yielded another doubling.)

For KeenType[2], a fork of the New Typesetting System (and more complete TeX implementation than JMathTeX), I added an SVG generator that converts numbers to string using a StackOverflow answer[3], instead of using Ryu[4]. The performance was even better and the algorithm vastly simpler.

Knuth was right: measure, then optimize.

[0]: https://github.com/jfree/jfreesvg

[1]: https://github.com/jfree/jfreesvg/pull/30

[2]: https://github.com/DaveJarvis/KeenType

[3]: https://stackoverflow.com/a/10554128/59087

[4]: https://github.com/DaveJarvis/KeenType/blob/fef005579021f394...

KeenType, a modernized and optimized NTS fork, replaces KeenTeX.

This gets styled in the PDF as:

https://i.ibb.co/ZfZXmDn/output.png

Various styles are packed in themes, which the user can select when exporting to PDF. At present, there are only three themes:

https://github.com/DaveJarvis/keenwrite-themes/tree/main/exa...

There other styles in the Boschet theme, such as speech bubbles, TODOs, and so forth:

    ::: bubbletx

KeenWrite Themes[1] are instructions that tell ConTeXt how to typeset XHTML documents (content) into PDF files (presentation). I made a tutorial that shows how my FOSS desktop text editor, KeenWrite[3], allows users to write in Markdown to typeset a document against a particular theme.

Before it can be used for scientific papers, it needs cross-references, which, unfortunately, aren't part of the CommonMark specification.

I posit that the vast majority of LaTeX users don't grok how to separate content from presentation. When I asked a question on TeX.SE about how to adjust the line spacing between enumerated items (spanning a couple dozen enumerated lists), the vast majority of people voted for the answer of using `\itemsep0em` to tweak each list ... individually.[4] The correct answer, IMO, is to fix the problem globally, and not waste time tweaking individual lists.

[1]: https://github.com/DaveJarvis/keenwrite-themes

[2]: https://www.youtube.com/watch?v=3QpX70O5S30

[3]: https://github.com/DaveJarvis/keenwrite

[4]: https://tex.stackexchange.com/questions/6081/reduce-space-be...

The ConTeXt typesetting system tightly integrates Lua. One aspect of Lua that I dislike is its inability to easily write OOP-ish code. What's impressive about the language is that it can be extended to do so in astonishingly little code:

* https://github.com/kikito/middleclass

With OOP in place, I was able to typeset a hexagonal grid and a symbolic representation of a neural network on top, using a more OOP-like approach. The classes are straightforward.

A vertex defines a point in 2D space:

* https://github.com/DaveJarvis/keenwrite-themes/blob/main/bos...

An edge connects two vertices:

* https://github.com/DaveJarvis/keenwrite-themes/blob/main/bos...

A graph connects edges:

* https://github.com/DaveJarvis/keenwrite-themes/blob/main/bos...

A priority queue serves for ordering graph edges by weight of adjoining vertices:

* https://github.com/DaveJarvis/keenwrite-themes/blob/main/bos...

With these concepts in hand, we can typeset a grid and a "neural network" on top:

* https://github.com/DaveJarvis/keenwrite-themes/blob/main/bos...

Here's an example of the output for chapter 1:

* https://i.ibb.co/19DCDZy/ch-1.png

And chapter 14, where the "network" has grown in complexity:

* https://i.ibb.co/ncf16vg/ch-2.png

This is for my near future hard sci-fi book on AGI. I'm looking for alpha readers to give me feedback. See profile for contact details.

KeenWrite uses themes to control every aspect of the presentation layer: leader dots, indentation levels, fonts, colours, hyperlinking, and so forth. This is accomplished by first converting R Markdown to XHTML then passing the XHTML to ConTeXt.

> like font face, font size, horizontal and vertical element positioning and line spacing, etc.

May I recommend ConTeXt?

For my purposes, KeenType was only meant to provide a real-time "rough draft" of equations rendered inside of KeenWrite[0], my FOSS Markdown editor. That is, I edit in Markdown, then KeenWrite converts the inline TeX-based equations for previewing. When I'm ready to create a "finished" product, KeenWrite exports the Markdown to XHTML then feeds the XHTML, along with a theme, to ConTeXt[1]. The various themes[2] are where font faces sizes, alignment, kerning, etc. are tweaked. In this way content remains completely separated from presentation.

[0]: https://github.com/DaveJarvis/keenwrite

[1]: https://wiki.contextgarden.net/Main_Page

[2]: https://github.com/DaveJarvis/keenwrite-themes

KeenWrite[0], the FOSS Markdown text editor I’ve been working on, includes the ability to render plain text diagrams via Kroki[1]†. See the screenshots[2] for examples. Here’s a sample Markdown document that was typeset[3] using ConTeXt[4] (and an early version of the Solare[5] theme).

One reason I developed KeenWrite was to use variables inside of plain text diagrams. In the genealogy diagram, when any character name (that’s within the diagram) is updated, the diagram regenerates automatically. (The variables are defined in an external YAML file, allowing for integration with build pipelines.)

Version 3.x containerizes the typesetting system, which greatly simplifies the installation instructions that allow typesetting Markdown into PDF files. It also opens the door to moving Kroki into the container so that diagram descriptions aren’t pushed over the Internet to be rendered.

†Kroki, ergo KeenWrite, supports BlockDiag (BlockDiag, SeqDiag, ActDiag, NwDiag, PacketDiag, RackDiag), BPMN, Bytefield, C4 (with PlantUML), Ditaa, Erd, Excalidraw, GraphViz, Nomnoml, Pikchr, PlantUML, Structurizr, SvgBob, UMLet, Vega, Vega-Lite, and WaveDrom.

Note that Mermaid diagrams generate non-conforming SVG[6], so they don’t render outside of web browsers. There is work being done to address[7] this problem.

[0]: https://github.com/DaveJarvis/keenwrite

[1]: https://kroki.io/

[2]: https://github.com/DaveJarvis/keenwrite/blob/main/docs/scree...

[3]: https://pdfhost.io/v/4FeAGGasj_SepiSolar_Highlevel_Software_...

[4]: https://wiki.contextgarden.net/Main_Page

[5]: https://github.com/DaveJarvis/keenwrite-themes/

[6]: https://github.com/mermaid-js/mermaid/issues/2485

[7]: https://github.com/yuzutech/kroki/issues/1410

KeenWrite leverages KeenWrite Themes to change how documents are presented. The Tarmes theme is an example of a very basic theme, meant to act as a base for making new themes. Take a look at Tarmes, which is probably the closest answer to your question. Feel free to add issues against the issue tracker or add questions to the discussion area.

> Nevertheless, I would prefer a Markdown-based system

My free, cross-platform desktop Markdown editor, KeenWrite[1], integrates with the ConTeXt typesetting software[2]. I'm working on a branch[3] to make integration containerized[3] because its installation is painful. KeenWrite limits math to plain TeX[4] so that the output can be rendered using any TeX-based typesetter (ConTeXt, LaTeX, MathJax, εχTEX, etc.).

Here's a sample document typeset using ConTeXt (skip to page 40 for the math):

https://pdfhost.io/v/4FeAGGasj_SepiSolar_Highlevel_Software_...

That document theme is called Solare[8].

> that can use CSS and MathML

Adding CSS mixes presentation logic with content, which is something KeenWrite strives to avoid. Instead, KeenWrite implements Pandoc's annotation syntax to keep presentation logic out of the content. I've written about this extensively in my Typesetting Markdown series[5].

You can produce some pretty amazing documents just with annotations, such as the following that I wrote in Markdown and typeset using ConTeXt:

https://impacts.to/downloads/lowres/impacts.pdf

> has a 100% bibtex clone for references.

Markdown fails at references. At some point, I'd like to implement cross-references in KeenWrite. Except there's at least six competing standards for the syntax, which I've also remarked upon[6], making the choice of syntax difficult[7].

[1]: https://github.com/DaveJarvis/keenwrite

[2]: https://wiki.contextgarden.net/Installation

[3]: https://github.com/DaveJarvis/keenwrite/blob/1_typeset_using...

[4]: https://github.com/DaveJarvis/keenwrite/blob/main/docs/scree...

[5]: https://dave.autonoma.ca/blog/2020/04/28/typesetting-markdow...

[6]: https://talk.commonmark.org/t/cross-references-and-citations...

[7]: https://xkcd.com/927/

[8]: https://github.com/DaveJarvis/keenwrite-themes/tree/main/sol...

If ConTeXt and KeenWrite Themes are installed, then you can press Control+P to export as a PDF file, producing the output shown on the far right.