janet-utf8 VS xi-editor

Regarding UTF-8 encoding:

“And a couple of important consequences:

- You CAN’T determine the length of the string by counting bytes.

- You CAN’T randomly jump into the middle of the string and start reading.

- You CAN’T get a substring by cutting at arbitrary byte offsets. You might cut off part of the character.”

One of the things I had to get used to when learning the programming language Janet is that strings are just plain byte sequences, unaware of any encoding. So when I call `length` on a string of one character that is represented by 2 bytes in UTF-8 (e.g. `ä`), the function returns 2 instead of 1. Similar issues occur when trying to take a substring, as mentioned by the author.

As much as I love the approach Janet took here (it feels clean and simple and works well with their built-in PEGs), it is a bit annoying to work with outside of the ASCII range. Fortunately, there are libraries that can deal with this issue (e.g. https://github.com/andrewchambers/janet-utf8), but I wish they would support conversion to/from UTF-8 out of the box, since I generally like Janet very much.

One interesting thing I learned from the article is that the first byte can always be determined from its prefix. I always wondered how you would recognize/separate a unicode character in a Janet string since it may have 1-4 bytes length, but I guess this is the answer.

Was confused until I realised I'd confused Zed, with Xi[1] which is also rust based, and which incidentally has a frontend called "Xim"..

Also there's a wiki-editor (like Tomboy[2]) called "Zim"[3].

[1] https://github.com/xi-editor/xi-editor

Project site linked from the GitHub[0] is https://xi-editor.io. Linked doc is a mirror of this[1], which was afaik originally written by Raph Linus.

[0]: https://github.com/xi-editor/xi-editor

[1]: https://xi-editor.io/docs/rope_science_01.html

> thing that gets deleted when you hit backspace

Is there a canonical source for this part, by the way? Xi copied the logic from Android[1] (as per the issue you linked downthread), and I vaguely remember that CLDR had something to say about this too, but I don’t know if there’s any sort of consensus here that’s actually written down anywhere.

[1] https://github.com/xi-editor/xi-editor/pull/837

This is pretty much how copy-on-write ropes work. Check out xi-rope, Ropey or crop, they're all built using B-trees and implement the behavior you described.

Helix is awesome, though once Lapce (spiritual successor to Xi editor) gets the Helix/Kakoune editing model, I may have to jump ship

Home (xi-editor.io)

Maybe take a look at the Xi editor (https://xi-editor.io/) (written in rust I think) that uses a client server architecture.

Have a look at what Lapce uses. The editor is coming along nicely, and iirc, they use the Xi editor as a plug-in.

Raph Levien posted a retrospective about using CRDT’s for collaborative editing in xi-editor here [1]. His conclusion is

“I come to the conclusion that the CRDT is not pulling its (considerable) weight. When I think about a future evolution of xi-editor, I see a much brighter future with a simpler, largely synchronous model, that still of course has enough revision tracking to get good results with asynchronous peers like the language server.”

[1]https://github.com/xi-editor/xi-editor/issues/1187#issuecomm...