talk VS willow

Also curious to hear about your setup. Using whisper too? When I was experimenting with it there was still a lot of annoyance about hallucinations and I was hard coding some "if last phrase is 'thanks for watching', ignore last phrase"

I was just googling a bit to see what's out there now for whisper/llama combos and came across this: https://github.com/yacineMTB/talk

There's a demo linked on the github page that seems relatively fast at responding conversationally, but still maybe 1-2 seconds at times. Impressive it's entirely offline.

Here’s a project attempting to do just this!

https://github.com/yacineMTB/talk

Funny, I just picked up a device for use with https://heywillow.io for similar reasons

Fair points but with all due respect completely misses the point and context. My comment was a reply to a new user interested in esphome on a post about esphome.

You're talking about CircuitPython, 35KB web replies, PSRAM, UF2 bootloader, etc. These are comparatively very advanced topics and you didn't mention esphome once.

The comfort and familiarity of Amazon for what is already a new, intimidating, and challenging subject is of immeasurable value for a novice. They can click those links, fill a cart, and have stuff show up tomorrow with all of the usual ease, friendliness, and reliability of Amazon. If they get frustrated or it doesn't work out they can shove it in the box and get a full refund Amazon-style.

You're suggesting wandering all over the internet, ordering stuff from China, multiple vendors, etc while describing a bunch of things that frankly just won't matter to them. I say this as someone who has been an esphome and home assistant user since day one. The approach I described has never failed or remotely bothered me and over the past ~decade I've seen it suggested to new users successfully time and time again.

In terms of PSRAM to my knowledge the only thing it is utilized for in the esphome ecosystem is higher resolution displays and more advanced voice assistant scenarios that almost always require -S3 anyway and are a very advanced, challenging use cases. I'm very familiar with displays, voice, the S3, and PSRAM but more on that in a second...

> live with one less LX7 core and no Bluetooth

I'm the founder of Willow[0] and when comparing Willow to esphome the most frequent request we get is supporting bluetooth functionality i.e. esphome bluetooth proxy[1]. This is an extremely popular use case in the esphome/home assistant community. Not having bluetooth while losing a core and paying more is a bigger issue than pin spacing.

It's also a pretty obscure board and while not a big deal to you and I if you look around at docs, guides, etc, etc you'll see the cheap-o boards from Amazon are by far the most popular and common (unsurprisingly). Another plus for a new user.

Speaking of Willow (and back to PSRAM again) even the voice assistant satellite functionality of Home Assistant doesn't fundamentally require it - the most popular device doesn't have it either[2].

Very valuable comment with a lot of interesting information, just doesn't apply to context.

[0] - https://heywillow.io/

[1] - https://esphome.io/components/bluetooth_proxy.html

[2] - https://www.home-assistant.io/voice_control/thirteen-usd-voi...

> - People might criticize my messy/bad/unfinished code

As someone who has created and maintained open source projects (most recently Willow[0]) for two decades I get a kick out of this.

Of course when interacting with users and feedback I keep it polite but in my head I'm thinking "You like to talk. I actually DID this. Shut up or submit a PR".

Surprise surprise they almost never do.

[0] - https://heywillow.io/

Also check out Willow- https://heywillow.io

It doesn’t synthesize voice back (yet) but open source and runs all offline on ESP32-based hardware and works with HomeAssistant!

Look into https://heywillow.io/ - still early in the project but they are getting good results.

Very nice!

Would you be interesting in integrating with my project Willow[0]?

Willow supports Home Assistant, OpenHAB, and generic REST+MQTT endpoints today. With Home Assistant and OpenHAB we benefit from their specific API support for providing speech to text output and processing through things like the HA Assist Pipelines[1].

From our standpoint we handle wake word, VAD+AEC+BSS, STT, TTS, user feedback, etc. All we really do is send the speech transcript to the Willow command endpoint (like HA) and speak+display the execution result. Other than all of the wild speech stuff and our obsession with speed and accuracy Willow is really quite "dumb" - think of it as a voice terminal.

OpenHAB has something similar but it's significantly more limited.

[0] - https://heywillow.io

[1] - https://developers.home-assistant.io/docs/voice/pipelines/

I'm the founder of Willow[0] (we use ctranslate2 as well) and I will be looking at this as soon tomorrow as these models are released. HF claims they're drop-in compatible but we won't know for sure until someone looks at it.

[0] - https://heywillow.io/

Shameless self-plug but with my project Willow[0] we have a management server implementation to deal with multiple devices, etc. We have a new feature called "Willow One Wake" that takes the incoming audio amplitude when wake word is detected and uses our Willow Application Server (python) to only activate wake on the device closest to the person speaking. Old and tired compared to the commercial stuff but a first in the open source space.

The asyncio improvements in Python 3.12 especially (plus perf generally) have been instrumental in enabling real world use of this. With Python 3.12 asyncio, uvloop, and FastAPI it works remarkably well[1]. As the demo video shows not only does it not delay responsiveness, it has granularity down to inches.

[0] - https://heywillow.io/

[1] - https://youtu.be/qlhSEeWJ4gs