cursorless VS community

What are your opinions on https://www.cursorless.org/ ?

Are you targeting developers?

My understanding was people who are serious about developing via voice use it pretty exclusively.

Like, yeah you need to learn commands, but "are often not worth it" feels like brushing a pretty massive offering under the rug.

Is learning vi / emacs commands not worth it (or shortcuts in another IDE?)

Is there a middle ground?

actually I just saw that someone is working on a keyboard mode for Cursorless! https://github.com/cursorless-dev/cursorless/issues?q=is%3Ai...

See for example: - https://youtu.be/xtOkYdwUves?si=X01vGNVhNRjj7kXh - https://www.cursorless.org/ - https://talonvoice.com/

I would highly recommend checking out https://www.cursorless.org/, an editor for voice built on Talon (what’s being used here by Josh) and a VSCode extension. If for nothing else than to watch the dev live code with it.

Yep. So I've been working for about fifteen years, had it from the start, but it just keep on getting worse. So now have migrated to a no keyboard solution, and am working on a no mouse solution.

My setup, software:

- talon (https://talonvoice.com/), basing my configuration on the standard repo (https://github.com/knausj85/knausj_talon/), with some minor modifications. This is how I navigate my whole computer, and runs the voice recognition and eye tracking. This is how I'm typing this right now. There are also a bunch of other little system changes that you need to make (eg, on macOS, disable double space to type a period), which is documented by the community with varying levels of success.

- cursorless (https://www.cursorless.org/), and then vscode. Before this I used Sublime Text for years and years, but cursorless is too good for coding. I know there are some projects on going to port it to different editors, but I haven't massively looked into it.

- vimium extension for web browsing (works in both chrome and firefox). This makes things like clicking links easier with your voice, without having too use an eye tracker all the time.

And then hardware:

- DPA 4488 microphone -> DAD6001 microdot-XLR converter -> Shure X2U USB XLR interface. ~£800. It's a very expensive microphone, but it's what the developer of talon uses (same brand anyway), and since I'm using this for work I want to remove every obstacle to having my voice recognised correctly and quickly

- Tobii Eye Tracker 5. ~£250. This is basically the only game in town. It works well enough but needs a lot of light. It also needs to be physically mounted to a monitor. In theory this means I don't need to use a mouse, but because I have a 34" ultrawide it doesn't work well on the edges, so I have a specific window size and location configured (say "layout mouse") which I could move apps into if I need to.

- At some point soon I want to replace the X2U with something higher quality and more permanent on my desk. Soon!

If you're programming, and open to doing it by voice, definitely check out Cursorless: https://www.cursorless.org/

It's worth mentioning Talon[0] here, which is a system for offline voice control as well, with great python-based scripting.

Using your computer or programming with it works like a charm, with some interesting and impressive projects like Cursorless[1] coming out as well, based on it.

[0]: https://talonvoice.com/

[1]: https://github.com/cursorless-dev/cursorless

Disclaimer: not affiliated, just a happy occasional user

Talon is what I would describe as accessibility for programmers (my definition, not the developer's!). You effectively write software that replaces keyboard and mouse usage, generically, flexibly and programmatically.

So when you write commands you bind them in different way: app specific[1], feature specific[2][3], OS-specific, hecking __programming language specific__[4] etc, and the in talon mixes and matches all of that stuff together.

So let's say I have VSCode focused on a javascript file . Talon knows this, and so I have "panel switch" which is a vscode specific command, and "op strict equal" to insert ` === `, but I also have generic text editing commands (because it's an editor), and multi cursor commands (because vscode has been tagged as multi cursor supporting), and tab commands (because vscode is a tab-based editor), and so on and so on.

If I then switched to the browser I would keep the generic text editing commands, and the tab commands, as it supports both of those things, but I would no longer have multi cursor support (or JS commands), because my browser doesn't support that.

This also means you can by and large use the same talon config (and so the same voice commands) on windows, mac and x11.

So for me switching to windows is actually less of a pain because most of the ways I interact with my computer don't actually change, as talon abstracts that away quite a bit.

[1] https://github.com/talonhub/community/blob/main/apps/vscode/... / https://github.com/talonhub/community/blob/main/apps/vscode/...

Yep. So I've been working for about fifteen years, had it from the start, but it just keep on getting worse. So now have migrated to a no keyboard solution, and am working on a no mouse solution.

My setup, software:

- talon (https://talonvoice.com/), basing my configuration on the standard repo (https://github.com/knausj85/knausj_talon/), with some minor modifications. This is how I navigate my whole computer, and runs the voice recognition and eye tracking. This is how I'm typing this right now. There are also a bunch of other little system changes that you need to make (eg, on macOS, disable double space to type a period), which is documented by the community with varying levels of success.

- cursorless (https://www.cursorless.org/), and then vscode. Before this I used Sublime Text for years and years, but cursorless is too good for coding. I know there are some projects on going to port it to different editors, but I haven't massively looked into it.

- vimium extension for web browsing (works in both chrome and firefox). This makes things like clicking links easier with your voice, without having too use an eye tracker all the time.

And then hardware:

- DPA 4488 microphone -> DAD6001 microdot-XLR converter -> Shure X2U USB XLR interface. ~£800. It's a very expensive microphone, but it's what the developer of talon uses (same brand anyway), and since I'm using this for work I want to remove every obstacle to having my voice recognised correctly and quickly

- Tobii Eye Tracker 5. ~£250. This is basically the only game in town. It works well enough but needs a lot of light. It also needs to be physically mounted to a monitor. In theory this means I don't need to use a mouse, but because I have a 34" ultrawide it doesn't work well on the edges, so I have a specific window size and location configured (say "layout mouse") which I could move apps into if I need to.

- At some point soon I want to replace the X2U with something higher quality and more permanent on my desk. Soon!

I don't have a disability but do game and play guitar to a degree that has damaged my body when combined with my day job, and use https://talonvoice.com/ with this configuration to do a lot of my typing in vim. https://github.com/knausj85/knausj_talon

The whole ecosystem around Talon is fantastic. You'll probably want to start with [0] if it's your first time, it's a community-maintained talon config which is a good starting point for customization.

The speech recognition engine built for it is great and the whole thing is free of charge, though I do recommend donating on Patreon [1].

Cursorless is a nice rethinking-from-the-ground kind of project. However, you can just as well keep using your current tools.

I've used Talon on and off with Jetbrains IDEs when I had some wrist pain and the overall experience was great, with my coding speed being fast enough without much practice. When voice typing, autocomplete is also very useful, so the introduction of Copilot was an additional huge improvemet.

But overall, the design of Talon, its config files, and the way this leads to very easy customizability, is really great. A bit like making your whole OS easily automatable and programmable (even if your OS isn't emacs /s).

[0]: https://github.com/knausj85/knausj_talon

[1]: https://www.patreon.com/lunixbochs

In fact, the most common (effectively the default) configuration for Talon includes a mouse grid https://github.com/knausj85/knausj_talon

Wow, Talon Voice looks like a great solution, and it seems like it has a pretty healthy open source community behind it!

I mainly use cursorless to accomplish this, along with knausj grammar files, all running on Talon. My setup could be a lot better but I have done projects since I lost much of the use of my hands.

Publish your talon commands on Github for others to use like https://github.com/knausj85/knausj_talon.

> I can find literally nothing online, or even internally, about using Visual Studio with Dragon NaturallySpeaking

My Talon project can do this (it can either use Dragon as a speech engine, or my own engine).

- https://talonvoice.com

- https://github.com/knausj85/knausj_talon/tree/master/apps/vs...

Here are a couple of public GitHub issues about dictation support in VSCode:

- https://github.com/microsoft/vscode/issues/40976

- https://github.com/microsoft/vscode/issues/117980#issuecomme...

I just had a quick try of Talon on Windows, using wav2letter (I don't have Dragon) and the recommended scripts[0]. It... doesn't work well for me at all.

I was able to get a couple of simple commands to work in Chrome, sometimes, such as "reload" and "show history". In Visual Studio code, it just spouted a bunch of errors in the console [1], and in JetBrains Rider all it would do it type gobbledygook, like a cat had walked on the keyboard or something. Pretty dissapointing :(

The logs also fill up with "WARNING actions: skipped because they have no matching declaration: (user.select_next_token)".

It was a bit confusing to use too (apart from not really working, I mean!), as it wasn't clear if I had to use some kind of command to enable voice commands, or if it was litening all the time. Eventually I figured out that it seems to be the latter, but still, it's not clear what commands it has heard and understood - I found myself speaking and nothing was happening, and I had no idea what it had understood. Similarly, I'd say something like "close tab", and it would type some nonsense like "aa&" into the current file - again, no idea what command it was actually trying to use.

[0] https://github.com/knausj85/knausj_talon