Top 7 C++ Utf8 Projects

ImGuiColorTextEdit

6 1,322 0.0 C++

Colorizing text editor for ImGui
simdutf

10 944 8.8 C++

Unicode routines (UTF8, UTF16, UTF32) and Base64: billions of characters per second using SSE2, AVX2, NEON, AVX-512, RISC-V Vector Extension. Part of Node.js and Bun.

Project mention: Vectorizing Unicode conversions on real RISC-V hardware | news.ycombinator.com | 2024-01-27

The project was mostly inspired by simdutf [0] which has been around for a couple of years already, and I don't think iconv has any of its vectorized implementations for other architectures.
[0] https://github.com/simdutf/simdutf
InfluxDB

www.influxdata.com
sponsored

Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
Rapidcsv

3 798 5.1 C++

C++ CSV parser library
tiny-utf8

2 534 0.0 C++

Unicode (UTF-8) capable std::string

Project mention: tiny-utf8 VS codepoint-iterator - a user suggested alternative | libhunt.com/r/tiny-utf8 | 2023-06-04
text

13 395 8.4 C++

Haskell library for space- and time-efficient operations over Unicode text. (by haskell)

Project mention: Super Colliding Nix Stores: Nix Flakes for Millions of Developers | news.ycombinator.com | 2023-05-25
uni-algo

4 241 8.9 C++

Unicode Algorithms Implementation for C/C++

Project mention: uni-algo: Unicode Algorithms Implementation for C/C++ | news.ycombinator.com | 2024-03-25
hypergrep

5 163 9.3 C++

Recursively search directories for a regex pattern

Project mention: Ugrep – a more powerful, ultra fast, user-friendly, compatible grep | news.ycombinator.com | 2023-12-30

Another issue with Hyperscan is that if you enable HS_FLAG_UTF8[1], which hypergrep does[2,3], and then search invalid UTF-8, then the result is UB.
> This flag instructs Hyperscan to treat the pattern as a sequence of UTF-8 characters. The results of scanning invalid UTF-8 sequences with a Hyperscan library that has been compiled with one or more patterns using this flag are undefined.
That's another issue you'll need to grapple with if you use Hyperscan. PCRE2 used to have this issue[4], but they've since defined the semantics of searching invalid UTF-8 with Unicode mode enabled. ripgrep 14 uses that new mode, but I haven't updated that FAQ answer yet.
[1]: https://intel.github.io/hyperscan/dev-reference/api_files.ht...
[2]: https://github.com/p-ranav/hypergrep/blob/ee85b713aa84e0050a...
[3]: https://github.com/p-ranav/hypergrep/blob/ee85b713aa84e0050a...
[4]: https://github.com/BurntSushi/ripgrep/blob/master/FAQ.md#why...
WorkOS

workos.com
sponsored

The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.

NOTE: The open source projects on this list are ordered by number of github stars. The number of mentions indicates repo mentiontions in the last 12 Months or since we started tracking (Dec 2020). The latest post mention was on 2024-03-25.