proposal-intl-segmenter VS grapheme-splitter

Splitting by grapheme clusters (or the characters the user actually sees): JS doesn't support this natively, so you'll need a library like grapheme-splitter. There's a Stage-4 proposal in the works, though: Intl.Segmenter:

Intl.Segmenter: Unicode segmentation in JavaScript slides.

But even that's not bulletproof. The best method is to divide the string into grapheme clusters before reversing, which is where Intl.Segmenter comes in.

Exactly, and emoji are outside the BMP, so it's not exactly an edge case, but the norm where two code units (UTF-16 double-bytes) are used to make one code point (Unicode character).

And it gets even worse, when you consider that for many purposes you're not even interested in code points but in graphemes -- e.g. a single visible emoji might actually be a combination of 5 code points, represented by 8 UTF-8 code units, taking up 16 bytes.

If you want to split a string by graphemes, you can either use the main dedicated library for it [3], or the relatively new API Intl.Segmenter [4] which is in Chrome and Safari, but still hasn't made it to Firefox [5].

[1] https://blog.jonnew.com/posts/poo-dot-length-equals-two

[2] https://www.contentful.com/blog/2016/12/06/unicode-javascrip...

[3] https://github.com/orling/grapheme-splitter

[4] https://github.com/tc39/proposal-intl-segmenter

[5] https://bugzilla.mozilla.org/show_bug.cgi?id=1423593

Also potentially (but not in practice so far) locale-specific. See the FAQ on Javascript's implementation: https://github.com/tc39/proposal-intl-segmenter#why-should-we-pass-a-locale-and-options-bag-for-grapheme-boundaries-isnt-there-just-one-way-to-do-it

Do note that if you're using an international language, you might want to check out Grapheme Splitter to divide strings into individual user perceived characters, as opposed to computer perceived characters. Since our text is in English, it'd just add unnecessary complication and an extra step to our project so I'm not adding it in :)

Splitting by grapheme clusters (or the characters the user actually sees): JS doesn't support this natively, so you'll need a library like grapheme-splitter. There's a Stage-4 proposal in the works, though: Intl.Segmenter:

Exactly, and emoji are outside the BMP, so it's not exactly an edge case, but the norm where two code units (UTF-16 double-bytes) are used to make one code point (Unicode character).

And it gets even worse, when you consider that for many purposes you're not even interested in code points but in graphemes -- e.g. a single visible emoji might actually be a combination of 5 code points, represented by 8 UTF-8 code units, taking up 16 bytes.

If you want to split a string by graphemes, you can either use the main dedicated library for it [3], or the relatively new API Intl.Segmenter [4] which is in Chrome and Safari, but still hasn't made it to Firefox [5].

[1] https://blog.jonnew.com/posts/poo-dot-length-equals-two

[2] https://www.contentful.com/blog/2016/12/06/unicode-javascrip...

[3] https://github.com/orling/grapheme-splitter

[4] https://github.com/tc39/proposal-intl-segmenter

[5] https://bugzilla.mozilla.org/show_bug.cgi?id=1423593

You’ll need a dependent package called GraphemeSplitter which is required if your font contains multi-byte characters like for ex. Hindi. The package is included in the above repo but it was created by orling.