DeepMenu VS PrecisionGazeMouse

I've been working on the menuing side [1] based on crossing Fitt's Law with Huffman trees. But, don't know the constraints for ALS.

Hopefully, whomever takes this on doesn't take the standard Accessibility approach, which is adding an extra layer of complexity on an existing UI.

A good friend, Gordon Fuller, found out he was going blind. So, he co-founded one of the first VR startups in the 90's. Why? For wayfinding.

What we came up with a concept of Universal design. Start over from first principles. Seeing Gordon use an Accessible UI is painful to watch, it takes three times as many steps to navigate and confirm. So, what is the factor? 0.3 X?

Imagine if we could refactor all apps with a LLM, and then couple it with an auto compete menu. Within that menu is personal history of all your past transversals.

What would be the result? A 10X? Would my sister in a wheelchair be able to use it? Would love to find out!

[1] https://github.com/musesum/DeepMenu

Am experimenting with a new menuing idiom with SwiftUI [1]. It is both delightful and infuriating. Am delighted by autolayout. But, way too much semantics for observers. To simplify, I intentionally avoided structs, for view models, and resorted to classes.

I hope to create a package and use it for metal based visual music synthesizer. But, those complaints about playing nice with UIKit? Am rather worried.

[1] https://github.com/musesum/DeepMenu

As someone who suffered some severe mobility impairment a few years ago and relied extensively on eye tracking for just over a year, https://precisiongazemouse.org/ (Windows) and https://talonvoice.com/ (multiplatform) are great. In my experience the hardware is already surprisingly good, in that you get accuracy to within an inch or half an inch depending on your training. Rather, it's all about the UX wrapped around it, as a few other comments have raised.

IMO Talon wins* for that by supporting voice recognition and mouth noises (think lip popping), which are less fatiguing than one-eye blinks for common actions like clicking. The creator is active here sometimes.

(* An alternative is to roll your own sort of thing with https://github.com/dictation-toolbox/dragonfly and other tools as I did, but it's a lot more effort)

I'm curious if anyone on here has ever tried eye-tracking as a way to move your mouse pointer around. For example product like this: https://precisiongazemouse.org/ (I just found it googling, I have no idea if it is any good)

Ah I see, I would definitely try other apps, Precision Gaze Mouse for example. I don't have any real experience with eye tracking, but I would imagine rigging up Talon the way you're thinking would make it less accurate than if it were on a 27 inch monitor.

You might like https://github.com/PrecisionGazeMouse/PrecisionGazeMouse or https://github.com/cmauri/eviacam - basically you control the mouse using your head via a webcam.