Top 23 machine-translation Open-Source Projects

• NLP-progress

  17 22,309 2.1 Python

  Repository to track the progress in Natural Language Processing (NLP), including the datasets and the current state-of-the-art for the most common NLP tasks.

• NeMo

  29 10,084 9.8 Python

  A scalable generative AI framework built for researchers and developers working on Large Language Models, Multimodal, and Speech AI (Automatic Speech Recognition and Text-to-Speech)

  

Project mention: [P] Making a TTS voice, HK-47 from Kotor using Tortoise (Ideally WaveRNN) | /r/MachineLearning | 2023-07-06

I don't test WaveRNN but from the ones that I know the best that is open source is FastPitch. And it's easy to use, here is the tutorial for voice cloning.

• InfluxDB

  www.influxdata.com featured

  Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

  

• espnet

  15 7,892 10.0 Python

  End-to-End Speech Processing Toolkit

  

Project mention: WhisperSpeech – An Open Source text-to-speech system built by inverting Whisper | news.ycombinator.com | 2024-01-17

You might check out this list from espnet. They list the different corpuses they use to train their models sorted by language and task (ASR, TTS etc):

https://github.com/espnet/espnet/blob/master/egs2/README.md

• OpenNMT-py

  6 6,574 8.7 Python

  Open Source Neural Machine Translation and (Large) Language Models in PyTorch

  

• manga-image-translator

  12 4,206 9.3 Python

  Translate manga/image 一键翻译各类图片内文字 https://cotrans.touhou.ai/

Project mention: [DISC] - The angel who came to pick me up is a Gal (Oneshot by Shiraishi Kouhei) | /r/manga | 2023-09-06

OCR works pretty good. ocr.space, ocr.best and cotrans.touhou.ai/ are all pretty nice.

• spark-nlp

  87 3,695 9.3 Scala

  State of the Art Natural Language Processing

  

Project mention: Spark NLP 5.1.0: Introducing state-of-the-art OpenAI Whisper speech-to-text, OpenAI Embeddings and Completion transformers, MPNet text embeddings, ONNX support for E5 text embeddings, new multi-lingual BART Zero-Shot text classification, and much more! | /r/Python | 2023-09-06

• argos-translate

  30 3,253 7.8 Python

  Open-source offline translation library written in Python

Project mention: Fast and secure translation on your local machine with a GUI | news.ycombinator.com | 2024-04-13

Interestingly, I think this is actually related to the offline translation features built into Firefox. Both are products of "Project Bergamot", but the Mozilla-maintained version was later merged into the Firefox application:

https://browser.mt/

https://blog.mozilla.org/en/mozilla/local-translation-add-on...

https://hacks.mozilla.org/2022/06/training-efficient-neural-...

https://github.com/mozilla/firefox-translations

https://firefox-source-docs.mozilla.org/toolkit/components/t...

Extra webpage with screenshot and links, impossible to search for normally:

https://translatelocally.com/downloads/

Does one thing and does it well.

Oh— For downloading models, it's much easier to pipe/`xargs` `translateLocally --available-models` into `translateLocally -d` than go through the GUI.

---

Other self-hostable translation tools:

https://www.apertium.org/index.eng.html

- Traditional rule-based translation. Seems to work pretty well, but no good desktop frontend.

https://www.argosopentech.com/

- Works, but crashy desktop app.

https://libretranslate.com/

- API wrapping Argos Translate.

https://lingva.thedaviddelta.com/

- Google Translate scraper/privacy frontend.

https://euroglot.com/

- Proprietary, subscription trialware.

• SaaSHub

  www.saashub.com featured

  SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives

  

• CTranslate2

  14 2,799 8.9 C++

  Fast inference engine for Transformer models

  

Project mention: Creando Subtítulos Automáticos para Vídeos con Python, Faster-Whisper, FFmpeg, Streamlit, Pillow | dev.to | 2024-04-29

• lingvo

  1 2,778 8.5 Python

  Lingvo

  

• RL4LMs

  5 2,094 0.0 Python

  A modular RL library to fine-tune language models to human preferences

  

Project mention: How To Setup a Model With Guardrails? | /r/LocalLLaMA | 2023-05-12

I think of guardrails as another dimension of human preferences: whether you are training a model to answer questions more gooder or avoid saying horrifying stuff, you are teaching the model a preference. So I thinks it's a straightforward RLHF problem but from a different perspective.

• BartyCrouch

  0 1,356 4.6 Swift

  Localization/I18n: Incrementally update/translate your Strings files from .swift, .h, .m(m), .storyboard or .xib files.

  

• fastText_multilingual

  1 1,186 0.0 Jupyter Notebook

  Multilingual word vectors in 78 languages

  

Project mention: Ask HN: What's the coolest non standard application of LLMs you've seen? | news.ycombinator.com | 2023-12-23

(6 years ago)

Aligning the fastText vectors of 78 languages

https://github.com/babylonhealth/fastText_multilingual/blob/...

• nematus

  1 796 0.0 Python

  Open-Source Neural Machine Translation in Tensorflow

  

• Opus-MT

  3 527 4.8 Python

  Open neural machine translation models and web services

  

• COMET

  3 400 8.0 Python

  A Neural Framework for MT Evaluation (by Unbabel)

  

• edenai-apis

  13 360 9.8 Python

  Eden AI: simplify the use and deployment of AI technologies by providing a unique API that connects to the best possible AI engines

  

Project mention: We're Building an Open-Source LLM/AI API Wrapper: Here's Why | news.ycombinator.com | 2023-08-28

HackerNoon featured our latest article in the "Future of AI" category

We explain how Eden AI contributes to the AI ecosystem in structuring AI and LLM APIs by creating the most accomplished Open-Source wrapper possible.

You can support us in reaching 1000 stars on Github here: https://github.com/edenai/edenai-apis

• dsnote

  1 332 9.8 C++

  Speech Note Linux app. Note taking, reading and translating with offline Speech to Text, Text to Speech and Machine translation.

Project mention: Speech Note: offline Linux app for note taking, reading and translating | news.ycombinator.com | 2023-08-30

• OPUS-MT-train

  1 302 1.7 Makefile

  Training open neural machine translation models

  

• bergamot-translator

  31 299 6.4 C++

  Cross platform C++ library focusing on optimized machine translation on the consumer-grade device.

  

Project mention: Fast and secure translation on your local machine with a GUI | news.ycombinator.com | 2024-04-13

Interestingly, I think this is actually related to the offline translation features built into Firefox. Both are products of "Project Bergamot", but the Mozilla-maintained version was later merged into the Firefox application:

https://browser.mt/

https://blog.mozilla.org/en/mozilla/local-translation-add-on...

https://hacks.mozilla.org/2022/06/training-efficient-neural-...

https://github.com/mozilla/firefox-translations

https://firefox-source-docs.mozilla.org/toolkit/components/t...

Extra webpage with screenshot and links, impossible to search for normally:

https://translatelocally.com/downloads/

Does one thing and does it well.

Oh— For downloading models, it's much easier to pipe/`xargs` `translateLocally --available-models` into `translateLocally -d` than go through the GUI.

---

Other self-hostable translation tools:

https://www.apertium.org/index.eng.html

- Traditional rule-based translation. Seems to work pretty well, but no good desktop frontend.

https://www.argosopentech.com/

- Works, but crashy desktop app.

https://libretranslate.com/

- API wrapping Argos Translate.

https://lingva.thedaviddelta.com/

- Google Translate scraper/privacy frontend.

https://euroglot.com/

- Proprietary, subscription trialware.

• bitextor

  2 279 5.9 Python

  Bitextor generates translation memories from multilingual websites

  

• pantran.nvim

  3 266 5.1 Lua

  Use your favorite machine translation engines without having to leave your favorite editor.

• masakhane-mt

  1 266 1.8 Lua

  Machine Translation for Africa

  

• cybertron

  1 258 7.2 Go

  Cybertron: the home planet of the Transformers in Go (by nlpodyssey)

  

• SaaSHub

  www.saashub.com featured

  SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives

  

machine-translation related posts

• Fast and secure translation on your local machine with a GUI

  6 projects | news.ycombinator.com | 13 Apr 2024

• Understand how transformers work by demystifying all the math behind them

  1 project | news.ycombinator.com | 4 Jan 2024

• [P] Why the Original Transformer Figure Is Wrong, And Some Other Interesting Tidbits

  2 projects | /r/MachineLearning | 27 May 2023

• Why the Original Transformer LLM Figure Is Wrong, and Other Interesting Tidbits

  2 projects | news.ycombinator.com | 24 May 2023

• ModernMT languages: Tosk Albanian vs Albanian

  1 project | /r/machinetranslation | 18 May 2023

• How To Setup a Model With Guardrails?

  3 projects | /r/LocalLLaMA | 12 May 2023

• Making a custom Google Translate equivalent / web translation filter for my conlang?

  2 projects | /r/conlangs | 26 Apr 2023
• A note from our sponsor - SaaSHub
  www.saashub.com | 2 May 2024

  SaaSHub helps you find the best software and product alternatives Learn more →

Index

Sponsored

SaaSHub - Software Alternatives and Reviews

SaaSHub helps you find the best software and product alternatives

www.saashub.com