icu
apertium
Our great sponsors
icu | apertium | |
---|---|---|
15 | 5 | |
2,476 | 83 | |
2.4% | - | |
9.7 | 6.2 | |
6 days ago | 2 months ago | |
C++ | C++ | |
GNU General Public License v3.0 or later | GNU General Public License v3.0 only |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
icu
- FreeBSD replaces bubblesort with mergesort on SYSINTs, results in 100x speed
-
Get wide characters one by one from a char * string
If you want graphemes, something lile libgrapheme might suit you. If just code points, then the utf8.h from ICU can work, but you can also roll your own if needed.
- Fatal error when I'm trying to build SQLite for Android with ICU enabled
-
Should golang create a money library on the exp package?
OP, is this money in the context of storing and counting money, or money in the context of internationalization and localization? For the latter, you probably want bindings to icu or the x/text/currency package.
-
Setting the TZ environment variable avoids thousands of system calls
https://github.com/unicode-org/icu/pull/2213
This affects all packages that have icu as a dependency, one of them being Node.js.
https://github.com/nodejs/node/issues/37271
I discovered this the hard way when some code malfunctioned shortly after daylight savings time kicked in.
-
PSA: Node 18.13+ changed date/time formatting
No clue! It seems like a very deep rabbit hole to go down. You can start here: https://github.com/unicode-org/icu Version 72 I believe is when this was changed.
-
Charlock_holmes installation error, Windows 11.
Installing icu4c from https://github.com/unicode-org/icu/releases/tag/release-71-1 and put the path in "--with-icu-lib"
-
libicui18n.so.58: cannot open shared object file
wget https://github.com/unicode-org/icu/archive/release-58-3.tar.gz
-
How to cout a non-ASCII character within a non-ASCII string
You need something like icu. https://github.com/unicode-org/icu
-
Me in the middle of a french conversation, when I forgot what gender a certain word is.
ICU, which ships as part of the core internationalization and localization frameworks of every major operating system, has a facility for transliterating keyboards! This is how you can type in Roman characters and get Devanagari in text, or enter pinyin and get Han characters. You can compose a Hiragana to Latin transliteration — but it’s a stretch to get actual English words this way.
apertium
-
Ask HN: Tell us about your project that's not done yet but you want feedback on
This is very cool, looking forward to it! I've been doing the same thing with Spanish Wikipedia articles for a while, using a few lines of Bash + Regex. I was using Apertium for it. https://apertium.org/ It's definitely worse than most ML-based solutions, but it works reliably and fast; you can run it entirely offline. With Spanish translations, the main problem I was facing is lack of vocabulary, so I created https://github.com/phil294/apertium-eng-spa-wiktionary which about doubles the amount of recognized words, albeit with wonky grammar.
-
Show HN: Unlimited machine translation API for $200 / Month
I used to keep track of the state of machine translation some years back.
I think the way you measure the success of an automated translation is edit distance, i.e. how many manual edits you need to make to a translated text before you reach some acceptable state. I suppose it's somewhat subjective, but it is possible to construct a benchmark and allow for multiple correct results.
The best resources I knew back then were:
VISL's CG-3 self-reported a competitively low edit distance compared to Google Translate: https://visl.sdu.dk/constraint_grammar.html -- the abstraction unfortunately requires a rather deep knowledge of any one particular language's grammar. It is a convincing argument that in order to beat Google Translate, you want less fuzzy machine learning and more structural analysis. But you also need a PhD in computational linguistics and deep knowledge of each language.
Apertium has an open-source pipeline: https://apertium.org/ -- seems to be much more like an open-source approach with a quality similar to Google Translate (although I don't know if it's better or worse; probably slightly worse in most cases, and with a slightly lower coverage).
-
Translating several languages into CV Creole
For context, I have been contributing CV Creole data to Unicode's CLDR and MediaWiki for a number of years now, but both are mostly manual work. I once considered setting up an Apertium language pair between CV Creole and Portuguese, given the grammatical similarities, but never got around to it.
-
"Lingva" Google Translate but without the tracking
Lingva is awesome. Also don't forget to check out LibreTranslate and Apertium. They are open source. Apertium can even translate web pages (you need to enter the URL).
-
How I installed Apertium on CentOS 7
#!/bin/bash set -x mkdir -p apertium-src && \ mkdir -p $MTDIR cd apertium-src && \ wget http://ftp.tsukuba.wide.ad.jp/software/gcc/releases/gcc-8.5.0/gcc-8.5.0.tar.gz -O - \ | gzip -dc \ | tar -xf - && \ cd gcc-8.5.0 && \ ./configure --prefix=$MTDIR --disable-multilib && \ make -j $(nproc) && \ make install && \ cd .. || exit 1 cd apertium-src && \ wget https://github.com/unicode-org/icu/releases/download/release-69-1/icu4c-69_1-src.tgz -O - \ | gzip -dc \ | tar -xf - \ && cd icu/source \ && CC=gcc CXX=g++ ./configure --prefix=$MTDIR \ && CC=gcc CXX=g++ make -j $(nproc) \ && CC=gcc CXX=g++ make install \ && cd ../.. \ || exit 1 cd apertium-src && \ svn checkout http://beta.visl.sdu.dk/svn/visl/tools/vislcg3/trunk vislcg3 && \ cd vislcg3 && ./get-boost.sh \ && ./cmake.sh -DCMAKE_INSTALL_PREFIX=$MTDIR \ -DICU_INCLUDE_DIR=$MTDIR/include \ -DICU_LIBRARY=$MTDIR/lib/libicuuc.so \ -DICU_IO_LIBRARY=$MTDIR/lib/libicuio.so \ -DICU_I18N_LIBRARY=$MTDIR/lib/libicui18n.so \ && make -j$(nproc) && \ make install && cd .. || exit 1 cd apertium-src && \ git clone https://github.com/apertium/lttoolbox && \ cd lttoolbox && ./autogen.sh --prefix=$MTDIR && make -j $(nproc) && make install && cd ../.. || exit 1 cd apertium-src && \ git clone https://github.com/apertium/apertium && \ cd apertium && ./autogen.sh --prefix=$MTDIR && make -j $(nproc) && make install && cd ../.. || exit 1 cd apertium-src && \ git clone https://github.com/apertium/apertium-lex-tools && \ cd apertium-lex-tools && ./autogen.sh --prefix=$MTDIR && make -j $(nproc) && make install && cd ../.. || exit 1 cd apertium-src && \ git clone https://github.com/apertium/apertium-tha && \ cd apertium-tha && ./autogen.sh --prefix=$MTDIR && make && make install && cd ../.. || exit 1 cd apertium-src && \ git clone https://github.com/apertium/apertium-tha-eng && \ cd apertium-tha-eng && ./autogen.sh --prefix=$MTDIR && make && make install && cd .. && \ cd .. || exit 1
What are some alternatives?
utfcpp - UTF-8 with C++ in a Portable Way
lingva-translate - Alternative front-end for Google Translate
harfbuzz - HarfBuzz text shaping engine
LibreTranslate - Free and Open Source Machine Translation API. Self-hosted, offline capable and easy to setup.
manticoresearch - Easy to use open source fast database for search | Good alternative to Elasticsearch now | Drop-in replacement for E in the ELK soon
lttoolbox - Finite state compiler, processor and helper tools used by apertium
MacOSX-SDKs - A collection of those pesky SDK folders: MacOSX10.1.5.sdk thru MacOSX11.3.sdk
apertium-tha-eng - Apertium translation pair for Thai and English
docker - Official docker for Manticore Search
apertium-tha - Apertium linguistic data for Thai
feature-express