htm.core VS bergamot-translator

OK I am still parsing the pre-print, but their approach seems very interesting. And equally interesting would be to see what happens if you apply HTM [1] to the same inputs model, _e.g._:

> We next applied our paradigm to a morphologically and electrically complex detailed biophysical compartmental model of a 3D reconstructed layer 5 cortical pyramidal cell (L5PC) from rat somatosensory cortex (Fig. 2A). The model is equipped with complex nonlinear membrane properties, a somatic spike generation mechanism and an excitable apical nexus capable of generating calcium spikes

[1] https://github.com/htm-community/htm.core

Interestingly, I think this is actually related to the offline translation features built into Firefox. Both are products of "Project Bergamot", but the Mozilla-maintained version was later merged into the Firefox application:

https://browser.mt/

https://blog.mozilla.org/en/mozilla/local-translation-add-on...

https://hacks.mozilla.org/2022/06/training-efficient-neural-...

https://github.com/mozilla/firefox-translations

https://firefox-source-docs.mozilla.org/toolkit/components/t...

Extra webpage with screenshot and links, impossible to search for normally:

https://translatelocally.com/downloads/

Does one thing and does it well.

Oh— For downloading models, it's much easier to pipe/`xargs` `translateLocally --available-models` into `translateLocally -d` than go through the GUI.

---

Other self-hostable translation tools:

https://www.apertium.org/index.eng.html

- Traditional rule-based translation. Seems to work pretty well, but no good desktop frontend.

https://www.argosopentech.com/

- Works, but crashy desktop app.

https://libretranslate.com/

- API wrapping Argos Translate.

https://lingva.thedaviddelta.com/

- Google Translate scraper/privacy frontend.

https://euroglot.com/

- Proprietary, subscription trialware.

It's possible with Neural Machine Translation models. Before being integrated into Firefox itself local translation was already available through the TranslateLocally add-on, see: https://browser.mt/

my understanding is this is using the same backend for the offline translator (aka Bergamot Project), it also supports online translation services (Google, DeepL, Bing, etc.)

Then you can run the OCR text through either a google, deepl, or one of the other commercial services to translate as a first pass. They all sell API access to the engines for bulk translation. Or you can use an open source engine like the new Bergamot Engine

It is running a full transformer type neural network on your cpu, it needs all the speed it can get.

It’s because the translation engine requires at least SSSE3.1 instructions[1]. These are translated to wasm SIMD instructions[2] which are only enabled by browsers if the underlying hardware is there to execute these at least somewhat efficient.

[1] https://github.com/browsermt/bergamot-translator/issues/418

[2] https://emscripten.org/docs/porting/simd.html

> But then you have to send your original comment to a translation company so there are privacy concerns there too.

There are modern offline translation systems available such as Project Bergamot https://browser.mt/

https://browser.mt/ ("Bergamot" was the technical name for the project.)

Already exists!

But it'd also be a good idea to check out https://translatelocally.com/ which is a UI for https://github.com/browsermt/bergamot-translator, used in Mozilla's offline MT browser extension. It's deep neural MT, but surpisingly undemanding (runs just fine on CPU's). Not yet in apt and such, but translatelocally at least has dpkg's and newest version should be usable from the CLI.

Consider Firefox translations, this is a completely offline machine translation extension based on this project.