Twitter Text Obj VS totally-safe-transmute

I recall that Twitter allowed 140 Chinese characters in tweets originally, but when they switched to 280 ascii characters, CJK languages were not included. The current documentation does say that some languages and emojis use 2 characters per symbol of the 280 limit, limiting those languages to 140. https://developer.twitter.com/en/docs/counting-characters

I assume you we’re going for half of the (new) character limit? That wouldn’t be bad as a rough guess, except Twitter uses UTF-8 encoding and what counts as a character is a bit complicated. Certain objects like usernames in a reply or image URLs hosted on Twitter are not counted, either. The maximum size of a modern tweet could therefore be upwards of 280 x 4 = 1,120 bytes.

Given the examples in this article it seems like it could potentially be used for this!

It turns out, working this out is non-trivial, as Twitter has a few rules around how it count's characters. These are basically:

Japanese, Korean and Chinese were excluded from that increased cap, because they already had a significant advantage in being able to put more information into a single character. (More specifically, it's implemented such that ideograms count as 2 characters now, and emoji do, too.)

The 280 character limit in a tweet isn't equal to the number of glyphs in a tweet.

https://developer.twitter.com/en/docs/counting-characters

For example, there is this (pure safe Rust) code: https://github.com/ben0x539/totally-safe-transmute/blob/main... which accesses external resources (/proc/self/mem) in order to violate the safety guarantees.

The first is the totally_safe_transmute crate. I mean, who wouldn't love library code that has .expect("welp") and .expect("oof") as its error handling? But that's not even the really scary part. Issue #2 ("i hate this") remains open to this day, but for obvious reasons there's no chance of resolution. This post has some context and a line-by-line explanation of how it works.

You mean like this?

You can also do that in rust on linux: https://github.com/ben0x539/totally-safe-transmute/blob/master/src/lib.rs

I want to correct this statement: Rust can be safer, but not if a library you use contains unsound code. Unsoundness is most often caused by unsafe code, but not always (totally_safe_transmute, anyone?). There is a misconception that unsafe code blocks are always unsound and should be avoided at all costs, but they're completely fine if the safety contracts are upheld. In fact, unsafe blocks isolate the potential issues to make it easier to identify where undefined behavior may be occurring. unsafe code blocks are a feature of the language, and their usage should not be viewed as opting out of any safety the language provides, imo.

Just reimplement totally_safe_transmute in Zig. No need for unsafe.

Why transmute() when you can totally_safe_transmute()?

Well, there is always the totally-safe-transmute.

They should use https://github.com/ben0x539/totally-safe-transmute