kaldi-active-grammar VS cursorless

https://github.com/dictation-toolbox/dragonfly

https://github.com/daanzu/kaldi-active-grammar

With an open system/engine, you can train your own personal speech model. For kaldi-active-grammar (https://github.com/daanzu/kaldi-active-grammar), you can do so without all that much difficulty, although the process/documentation could certainly use improvement.

I bootstrapped my personal speech model by retaining the commands from me using WSR. My voice is quite abnormal, and it took only 10 hours of speech data to train a model orders of magnitude more accurate than any generic model I've ever used. And of course, I retain much of my usage now with Kaldi, so my model improves more and more over time. A virtuous flywheel!

I have been coding entirely by voice for approximately 10 years now (by hand long before that). Most of that time I have been using the Dragonfly (https://github.com/dictation-toolbox/dragonfly) library to construct my own customized voice coding system. The library is highly flexible and open source, allowing you to easily customize everything to suit what you need to be productive. It is perhaps the power user analogue to Dragon Naturally Speaking. With it, you can certainly be highly productive coding by voice. In fact, I develop kaldi-active-grammar (https://github.com/daanzu/kaldi-active-grammar), a free and open source speech recognition backend usable by Dragonfly, itself entirely by voice. There's also a community of voice coders using Dragonfly and other tools that build on top of it, such as Caster (https://github.com/dictation-toolbox/Caster).

- Demo: https://www.youtube.com/watch?v=Qk1mGbIJx3s / Software: https://github.com/daanzu/kaldi-active-grammar

Far field audio is usually harder for any speech system to get correct, so having a good quality mic and using it nearby will _usually_ help with the transcription quality. As a long time Linux user, I would love to see it get some more powerful voice tools - really hope that this opens up over the next few years. Feel free to drop me an email (on my profile) happy to help with setup on any of the above.

kaldi repo: https://github.com/daanzu/kaldi-active-grammar

What are your opinions on https://www.cursorless.org/ ?

Are you targeting developers?

My understanding was people who are serious about developing via voice use it pretty exclusively.

Like, yeah you need to learn commands, but "are often not worth it" feels like brushing a pretty massive offering under the rug.

Is learning vi / emacs commands not worth it (or shortcuts in another IDE?)

Is there a middle ground?

actually I just saw that someone is working on a keyboard mode for Cursorless! https://github.com/cursorless-dev/cursorless/issues?q=is%3Ai...

See for example: - https://youtu.be/xtOkYdwUves?si=X01vGNVhNRjj7kXh - https://www.cursorless.org/ - https://talonvoice.com/

I would highly recommend checking out https://www.cursorless.org/, an editor for voice built on Talon (what’s being used here by Josh) and a VSCode extension. If for nothing else than to watch the dev live code with it.

Yep. So I've been working for about fifteen years, had it from the start, but it just keep on getting worse. So now have migrated to a no keyboard solution, and am working on a no mouse solution.

My setup, software:

- talon (https://talonvoice.com/), basing my configuration on the standard repo (https://github.com/knausj85/knausj_talon/), with some minor modifications. This is how I navigate my whole computer, and runs the voice recognition and eye tracking. This is how I'm typing this right now. There are also a bunch of other little system changes that you need to make (eg, on macOS, disable double space to type a period), which is documented by the community with varying levels of success.

- cursorless (https://www.cursorless.org/), and then vscode. Before this I used Sublime Text for years and years, but cursorless is too good for coding. I know there are some projects on going to port it to different editors, but I haven't massively looked into it.

- vimium extension for web browsing (works in both chrome and firefox). This makes things like clicking links easier with your voice, without having too use an eye tracker all the time.

And then hardware:

- DPA 4488 microphone -> DAD6001 microdot-XLR converter -> Shure X2U USB XLR interface. ~£800. It's a very expensive microphone, but it's what the developer of talon uses (same brand anyway), and since I'm using this for work I want to remove every obstacle to having my voice recognised correctly and quickly

- Tobii Eye Tracker 5. ~£250. This is basically the only game in town. It works well enough but needs a lot of light. It also needs to be physically mounted to a monitor. In theory this means I don't need to use a mouse, but because I have a 34" ultrawide it doesn't work well on the edges, so I have a specific window size and location configured (say "layout mouse") which I could move apps into if I need to.

- At some point soon I want to replace the X2U with something higher quality and more permanent on my desk. Soon!

If you're programming, and open to doing it by voice, definitely check out Cursorless: https://www.cursorless.org/

It's worth mentioning Talon[0] here, which is a system for offline voice control as well, with great python-based scripting.

Using your computer or programming with it works like a charm, with some interesting and impressive projects like Cursorless[1] coming out as well, based on it.

[0]: https://talonvoice.com/

[1]: https://github.com/cursorless-dev/cursorless

Disclaimer: not affiliated, just a happy occasional user