noise VS esp-sr

This is Noise IK (possibly with minor differences in the hashing):

https://noiseprotocol.org/

Wireguard uses NoiseIK, plus a static public key for the initiator which is encrypted to the agreed-upon-session-key without adding additional round trips. Your protocol simply omits the parts related to the initiator's static public key, because it has none.

With regard to this:

> - On the wire/protocol stuff. We're doing pretty rudimentary "open new connection, stream voice, POST somewhere". This adds extra latency and CPU usage because of repeated TLS handshakes, etc. We have plans to use Websockets and what-not to cut down on this.

I've recently used the Noise protocol[1] to do some encrypted communication between two services I control but separated by the internet.

It was surprisingly easy!

[1]: https://noiseprotocol.org/

Rolling your own initial handshake is hard. Right now I strongly encourage you take a look at the Noise protocol framework. Specifically the XK and IK patterns for identified clients, and the NK pattern for anonymous clients. The best security will be achieved by the XK pattern, but if you need to reduce the number of messages to a minimum IK might be a bit more attractive. (Also, if I recall correctly IK is used by Wireguard, so there's an example to follow).

Rosenpass author here;

There is a confusion about terminology here I think. Mathematical proofs including cryptography proofs use models simplifying reality; i.e. the real practical system might still be susceptible to attacks despite a proof of security.

For crypto primitives (classic mc eliece, curve25519, ed25519, RSA, etc etc) the standard for proofs is currently showing that they are as hard as some well studied mathematical problem. This is done by showing that an attack on the primitive leads to an attack on the underlying mathematical primitive. The proof for Diffie-Hellman shows that attacking DH leads to an efficient solution for the discrete log problem. I.e. the proof is a reduction to the underlying primitive.

No primitive is perfectly secure (at least a brute force – i.e. guessing each possibility is possible); there is some probability that the adversary can guess the right key. We call this probability the adversary's advantage. One task in cryptoanalysis is to find better attacks against primitives with a higher advantage; if an attack with a polynomial time average runtime is found, the primitive is broken. Finding a higher non-polynomial attack is still an interesting result.

The standard for protocols is proving that the protocol is secure assuming the primitives are secure; since multiple primitives are used you basically get a formula deriving an advantage for breaking the entire protocol. The proof is a reduction to a set of primitives.

We did not build a proof in that gold standard, although we are working on it. We built a proof in the symbolic model – known as a symbolic analysis. This uses the perfect cryptography assumption; i.e. we assumed that the advantages for each primitive are zero. Google "Dolev-Yao-Model".

This makes the proof much easier; a proof assistant such as ProVerif can basically find a proof automatically using logic programming methods (horn clauses).

The definitions of security are fairly well understood; unfortunately there is a lot to go into so I can't expand on that here. Looking up "IND-CPA" and "IND-CCA" might be a good start; these are the security games/models of security for asymmetric encryption; you could move on to the models for key exchange algorithms there. Reading the [noise protocol spec](https://noiseprotocol.org/) is also a good start.

There is no description of the protocol or of its security goals, so I am making some guesses based on a cursory look at the source and what I imagine this might be for.

A single symmetric key is derived for both directions, and there is no checking of nonces, so as far as I can tell any message can be dropped, reordered, or replayed in both directions. (Including replaying message from A to B as if they were from B to A.)

This is a bit like using ECB and likely to lead to fun application-specific attacks like [0].

This is very much rolling your own crypto, in a dangerous way. I am on the record as being "against" the "don't roll your own crypto" refrain [1], but mostly because it doesn't work: it should discourage people from publishing hand-rolled protocols such as this, but instead people think it means "don't roll your own primitives" and accept the use of "Ed25519/X25519" as probably secure.

Please read about the Noise framework [2] to get an idea of how much nuance there is to this, and consider using a Go implementation of it [3] instead.

P.S. This kind of issue is also why I maintain that NaCl is not a high-level scheme [4]: this could have used NaCl and have the exact same issues. libsodium has a couple slightly higher-level APIs that could have helped, secretstream [5] and kx [6], but again please use Noise.

[0] https://cryptopals.com/sets/2/challenges/13

[1] https://securitycryptographywhatever.buzzsprout.com/1822302/...

[2] https://noiseprotocol.org/noise.html

[3] https://github.com/flynn/noise

[4] https://words.filippo.io/dispatches/nacl-api/

[5] https://libsodium.gitbook.io/doc/secret-key_cryptography/sec...

[6] https://libsodium.gitbook.io/doc/key_exchange

If it is the right thing, yes: https://github.com/espressif/esp-sr

For wake word and voice activity detection, audio processing, etc we use the ESP SR (speech recognition) framework from Espressif[0].

For speech to text there are two options and more to come:

1) Completely on device command recognition using the ESP SR Multinet 6 model. Willow will (currently) pull your light and switch entities from Home Assistant and generate the grammar and command definition required by Multinet. We want to develop a Willow Home Assistant component that will provide tighter Willow integration with HA and allow users to do this point and click with dynamic updates for new/changed entities, different kinds of entities, etc all in the HA dashboard/config.

The only "issue" with Multinet is that it only supports 400 defined commands. You're not going to get something like "What's the weather like in $CITY?" out of it.

For that we have:

2-?) Our own highly optimized inference server using Whisper, LLamA/Vicuna, and Speecht5 from transformers (more to come soon). We're open sourcing it next week. Willow streams audio after wake in realtime, gets the STT output, and sends it wherever you want. With the Willow Home Assistant component (doesn't exist yet) it will sit in between our inference server implementation doing STT/TTS or any other STT/TTS implementation supported by Home Assistant and handle all of this for you.

[0] - https://github.com/espressif/esp-sr

I don't believe that's possible. Seems like a pretty delicate process in the readme...