egui VS accesskit

Anyone who's interested in the AccessKit integration can play with my work-in-progress branch: https://github.com/mwcampbell/egui/tree/accesskit

It's currently Windows-only, and I'm working on the big missing feature, which is text editing support.

> A core premise of Cocoa, and MVC in general, is that UIs are a projection of data into a different form of data, specifically bits on a screen.

This is a tangent, but the implicit assumption that the UI is visual is just begging for a response from an accessibility perspective, so here goes.

Accessibility is very much an afterthought in native GUIs, not only in Cocoa, but also in Windows with the UI Automation API, and AFAIK with other native accessibility APIs as well. With these APIs, the assistive technology (e.g. screen reader) pulls information from the application (usually via the GUI toolkit), through repeated calls to methods defined by the accessibility API. Often the AT has to do several such calls in a row (and those often translate to multiple IPC round trips, making things slow). And the UI might change between such calls; there's no guaranteed way to get a consistent snapshot of the whole thing, as there is with a visual frame. On the application/toolkit side, these methods may return different responses from one call to the next, and the application or toolkit has to fire the right events when things change.

The web improves on this, in that accessibility information is conveyed through HTML tags and attributes. And yes, this is included in the output of a React component's render function. So while in practice, implementing accessibility may still be an afterthought, it's not an architectural afterthought as it is in native platforms.

One of my goals in AccessKit [1] is to work around this shortcoming of native accessibility APIs, particularly for developers of cross-platform non-web GUI toolkits. In AccessKit, the toolkit pushes a full or incremental accessibility tree update to the AccessKit platform adapter, which maintains the full tree in memory and uses that to implement the platform accessibility API. This even works for immediate-mode GUIs, as one can see in my proof-of-concept integration with the Rust egui toolkit [2].

[1]: https://github.com/AccessKit/accesskit

[2]: https://github.com/mwcampbell/egui/tree/accesskit

I can also report some modest progress on my own work on accessibility of immediate-mode GUIs. I have a branch of the Rust egui library [1] that has basic accessibility on Windows using my AccessKit project [2]. I do have a long way to go to make this fully usable and ready to submit upstream, especially when taking non-Windows platforms into account.

[1]: https://github.com/mwcampbell/egui/tree/accesskit

[2]: https://github.com/AccessKit/accesskit

If you were to implement this yourself, i'd look into either swash or cosmic-text for the text rendering stack (this is one of the things you really don't want to write from the ground up). For accessibility, AccessKit has quickly become the standard for communicating with crossplatform accessibility APIs in rust GUI. lightningcss (or its lower level counterpart cssparser) are both decent options for CSS parsing. Taffy handles some of what browsers offer for a layout engine, but is still being worked on.

> Fleet relies on the Java AWT/Swing framework to get a window from an operating system, but it doesn’t use the Java platform for managing its GUI components besides one JFrame and JPanel on top of it.

This is a terrible decision that is going to bite them in the long run. Doing things this way makes it far, far more difficult to implement accessibility, and regulations on this are only going to get stricter.

Implementing accessibility for a framework like that would involve three separate implementations for three separate platforms and the need to interface with D-Bus, COM and Objective C, from Java. I imagine that the latter two would be particularly difficult, considering how bad Java's FFI support is. It's not just calling methods either, you'd actually need to implement your own classes that conform to the relevant COM interfaces / Objective C protocols. There are libraries that can help with this[1], but I don't think they would work particularly well for something as complex as a code editor.

[1] https://github.com/AccessKit/accesskit

fltk-accesskit is an accesskit integration crate for fltk-rs, the gui crate. It's implemented as an external crate to allow for more experimentation before stabilizing the api, especially since fltk is at version 1.4.

Libraries for a lot of this stuff exist (albeit in many cases not very mature yet):

- https://github.com/pop-os/cosmic-text does text layout (which Taffy explicitly considers out of scope)

- https://github.com/AccessKit/accesskit does accessibility

- https://github.com/servo/rust-cssparser does value-agnostic CSS parsing (it will parse the general syntax but leaves value parsing up to the user, meaning you can easily add support for whatever properties you what). Libraries like https://github.com/parcel-bundler/lightningcss implement parsing for the standard css properties.

- There are crates like https://github.com/BurntSushi/bstr and https://docs.rs/wtf8/latest/wtf8/ for working with non-unicode text

We are planning to add a C API to Taffy, but tbh I feel like C is not very good for this kind of modularised approach. You really want to be able to expose complex APIs with enforced type safety and this isn't possible with C.

There are a number of up-and-coming Rust-based frameworks in this niche:

- https://github.com/iced-rs/iced (probably the most usable today)

- https://github.com/vizia/vizia

- https://github.com/marc2332/freya

- https://github.com/linebender/xilem (currently very incomplete but exciting because it's from a team with a strong track record)

What is also exciting to me is that the Rust GUI ecosystem is in many cases building itself up with modular libraries. So while we have umpteen competing frameworks they are to a large degree all building and collaborating on the same foundations. For example, we have:

- https://github.com/rust-windowing/winit (cross-platform window creation)

- https://github.com/gfx-rs/wgpu (abstraction on top of vulkan/metal/dx12)

- https://github.com/linebender/vello (a canvas like imperative drawing API on top of wgpu)

- https://github.com/DioxusLabs/taffy (UI layout algorithms)

- https://github.com/pop-os/cosmic-text (text rendering and editing)

- https://github.com/AccessKit/accesskit (cross-platform accessibility APIs)

In many cases there a see https://blessed.rs/crates#section-graphics-subsection-gui for a more complete list of frameworks and foundational libraries)

Using HarfBuzz makes sense. But if you're looking for a pure-Rust alternative, I hear cosmic-text (made by Pop!_OS) is good. There's also AccessKit for accessibility.

There are efforts to support a cross platform accessibility library:

https://github.com/AccessKit/accesskit

egui is an easy-to-use immediate mode GUI for Rust, and I just released 0.20. It's a big release!

There is now support for AccessKit (https://github.com/AccessKit/accesskit) which brings accessibility to egui (a first for an immediate mode GUI?).

There is also better table support, nicer keyboard shortcut handling, better looking text in light mode (still not great, but better).

See more in the changelog: https://github.com/emilk/egui/blob/master/CHANGELOG.md

Try it out at www.egui.rs