apertium VS KeenWrite

This is very cool, looking forward to it! I've been doing the same thing with Spanish Wikipedia articles for a while, using a few lines of Bash + Regex. I was using Apertium for it. https://apertium.org/ It's definitely worse than most ML-based solutions, but it works reliably and fast; you can run it entirely offline. With Spanish translations, the main problem I was facing is lack of vocabulary, so I created https://github.com/phil294/apertium-eng-spa-wiktionary which about doubles the amount of recognized words, albeit with wonky grammar.

I used to keep track of the state of machine translation some years back.

I think the way you measure the success of an automated translation is edit distance, i.e. how many manual edits you need to make to a translated text before you reach some acceptable state. I suppose it's somewhat subjective, but it is possible to construct a benchmark and allow for multiple correct results.

The best resources I knew back then were:

VISL's CG-3 self-reported a competitively low edit distance compared to Google Translate: https://visl.sdu.dk/constraint_grammar.html -- the abstraction unfortunately requires a rather deep knowledge of any one particular language's grammar. It is a convincing argument that in order to beat Google Translate, you want less fuzzy machine learning and more structural analysis. But you also need a PhD in computational linguistics and deep knowledge of each language.

Apertium has an open-source pipeline: https://apertium.org/ -- seems to be much more like an open-source approach with a quality similar to Google Translate (although I don't know if it's better or worse; probably slightly worse in most cases, and with a slightly lower coverage).

For context, I have been contributing CV Creole data to Unicode's CLDR and MediaWiki for a number of years now, but both are mostly manual work. I once considered setting up an Apertium language pair between CV Creole and Portuguese, given the grammatical similarities, but never got around to it.

Lingva is awesome. Also don't forget to check out LibreTranslate and Apertium. They are open source. Apertium can even translate web pages (you need to enter the URL).

#!/bin/bash set -x mkdir -p apertium-src && \ mkdir -p $MTDIR cd apertium-src && \ wget http://ftp.tsukuba.wide.ad.jp/software/gcc/releases/gcc-8.5.0/gcc-8.5.0.tar.gz -O - \ | gzip -dc \ | tar -xf - && \ cd gcc-8.5.0 && \ ./configure --prefix=$MTDIR --disable-multilib && \ make -j $(nproc) && \ make install && \ cd .. || exit 1 cd apertium-src && \ wget https://github.com/unicode-org/icu/releases/download/release-69-1/icu4c-69_1-src.tgz -O - \ | gzip -dc \ | tar -xf - \ && cd icu/source \ && CC=gcc CXX=g++ ./configure --prefix=$MTDIR \ && CC=gcc CXX=g++ make -j $(nproc) \ && CC=gcc CXX=g++ make install \ && cd ../.. \ || exit 1 cd apertium-src && \ svn checkout http://beta.visl.sdu.dk/svn/visl/tools/vislcg3/trunk vislcg3 && \ cd vislcg3 && ./get-boost.sh \ && ./cmake.sh -DCMAKE_INSTALL_PREFIX=$MTDIR \ -DICU_INCLUDE_DIR=$MTDIR/include \ -DICU_LIBRARY=$MTDIR/lib/libicuuc.so \ -DICU_IO_LIBRARY=$MTDIR/lib/libicuio.so \ -DICU_I18N_LIBRARY=$MTDIR/lib/libicui18n.so \ && make -j$(nproc) && \ make install && cd .. || exit 1 cd apertium-src && \ git clone https://github.com/apertium/lttoolbox && \ cd lttoolbox && ./autogen.sh --prefix=$MTDIR && make -j $(nproc) && make install && cd ../.. || exit 1 cd apertium-src && \ git clone https://github.com/apertium/apertium && \ cd apertium && ./autogen.sh --prefix=$MTDIR && make -j $(nproc) && make install && cd ../.. || exit 1 cd apertium-src && \ git clone https://github.com/apertium/apertium-lex-tools && \ cd apertium-lex-tools && ./autogen.sh --prefix=$MTDIR && make -j $(nproc) && make install && cd ../.. || exit 1 cd apertium-src && \ git clone https://github.com/apertium/apertium-tha && \ cd apertium-tha && ./autogen.sh --prefix=$MTDIR && make && make install && cd ../.. || exit 1 cd apertium-src && \ git clone https://github.com/apertium/apertium-tha-eng && \ cd apertium-tha-eng && ./autogen.sh --prefix=$MTDIR && make && make install && cd .. && \ cd .. || exit 1

KeenWrite is my free, open-source, cross-platform desktop Markdown editor that can produce beautifully typeset PDFs. I started working on it years ago to help write a novel that has a complex timeline and I couldn't find a text editor that would allow me to integrate a character sheet with the story itself.

https://github.com/DaveJarvis/keenwrite

Tutorials:

* https://www.youtube.com/playlist?list=PLB-WIt1cZYLm1MMx2FBG9...

Here's what I mean by using variables directly:

* https://www.youtube.com/watch?v=CFCqe3A5dFg

CommonMark doesn't propose a standard for bibliographic references. Would anyone find the editor more appealing if it had cross-references and citations?

My cross-platform desktop text editor, KeenWrite, allows users to define variables in an external YAML file. The editor calls out to Kroki[1] to convert text-based diagrams to SVG. The diagrams can reference variables and are rendered using EchoSVG[2].

KeenWrite[3] can produce PDF documentation from Markdown documents that has PlantUML diagrams with elements stored in an external, machine-readable file. Here are screenshots showing variables on the left, diagram text in the middle, and a real-time render on the right:

* https://raw.githubusercontent.com/DaveJarvis/KeenWrite/main/...

* https://raw.githubusercontent.com/DaveJarvis/KeenWrite/main/...

KeenWrite supports all diagrams offered by Kroki, which includes "diagram-plantuml".

[1]: https://kroki.io/

[2]: https://github.com/css4j/echosvg/

[3]: https://github.com/DaveJarvis/keenwrite

Although not interactive, I've created a video series that shows advanced usage of Markdown. Namely R, external variables, diagrams, math, annotations, and a different approach to metadata:

* https://www.youtube.com/playlist?list=PLB-WIt1cZYLm1MMx2FBG9...

Tutorial 4 shows basic Markdown:

* https://www.youtube.com/watch?v=qNbGSiRzx-0

The top-right of each video shows keyboard and mouse clicks to help follow along.[1] My desktop text editor, KeenWrite[2], is used in the tutorials.

[1]: https://github.com/DaveJarvis/kmcaster

[2]: https://github.com/DaveJarvis/keenwrite

https://github.com/DaveJarvis/keenwrite/blob/main/scripts/bu...

My template script provides a way to make user-friendly shell scripts. In a script that uses the template, you define the dependencies and their sources:

    DEPENDENCIES=(

I didn't create the fork, nor am I affiliated with the project. I use it in my text editor, KeenWrite to rasterize SVG.

KeenWrite Themes[1] are instructions that tell ConTeXt how to typeset XHTML documents (content) into PDF files (presentation). I made a tutorial that shows how my FOSS desktop text editor, KeenWrite[3], allows users to write in Markdown to typeset a document against a particular theme.

Before it can be used for scientific papers, it needs cross-references, which, unfortunately, aren't part of the CommonMark specification.

I posit that the vast majority of LaTeX users don't grok how to separate content from presentation. When I asked a question on TeX.SE about how to adjust the line spacing between enumerated items (spanning a couple dozen enumerated lists), the vast majority of people voted for the answer of using `\itemsep0em` to tweak each list ... individually.[4] The correct answer, IMO, is to fix the problem globally, and not waste time tweaking individual lists.

[1]: https://github.com/DaveJarvis/keenwrite-themes

[2]: https://www.youtube.com/watch?v=3QpX70O5S30

[3]: https://github.com/DaveJarvis/keenwrite

[4]: https://tex.stackexchange.com/questions/6081/reduce-space-be...