nnnoiseless VS nih-plug

1. So the SFUs we're currently looking at are yours, ion-sfu (and/or galene) and mediasoup. Honestly we haven't finished looking at how they compare for rate control, but the Pion team seems very interested in ensuring they have good rate control.

2. From context I think you're talking about noise cancellation here? I assumed that some of the more exotic ML-based ones ran serverside, which obviously is incompatible with E2EE. It sounds like there are a bunch of options for running WASM-based intelligent noise cancellation clientside though, especially with MediaStreamTrackProcessor and friends. rnnoiseless as a pure Rust->WASM port of rnnoise looks fun, for instance: https://github.com/jneem/nnnoiseless

3. True, although given Google are highly motivated to make AEC work properly in WebRTC, I guess I'm hoping that they'll continue improving it, much as they have been. I certainly never want to have to write or integrate one ever again :D

Very nice! I've just updated [`nnnoiseless`](https://github.com/jneem/nnnoiseless) to rustfft 5.0, and got a cool 20% overall improvement.

Super cool. Any experience doing audio/synthesis/DSP work with this in a Flutter app? It would be particularly awesome if this enabled building VST plugins with Flutter and one of the Rust crates for VSTs (like NIH-plug or similar).

Looks nice! Rust really is a perfect fit for real-time audio software. At work we also use cpal for audio output, and has found some bugs as well. In my free time I'm tinkering with CLAP plugins for Bitwig, your app looks quite similar to Bitwig Grid which I like a lot. Maybe providing it as a plugin would be a future option.

I don't think Max4Live is not a good choice for building audio plugins. It's a weird platform that was designed for 'institutionalized academic music,' as I once read someone describe it. It's difficult to program in and not efficient. None of my favorite music software is made with it. It's also quite buggy, in my experience. For doing some basic extensions to Ableton Live specifically, beyond what VST allows access to, it's OK, since it's the only official way to do so.

If you want to just dive into DSP using wires and boxes, with some additional code sprinkled in, SynthEdit or Reaktor Core are faster, more fun, and produce better results. If you don't mind C++, check out iPlug from REAPER's WDL codebase: https://www.cockos.com/wdl/ — there are some forks of it.

There's also JUCE. You'll find some people complain about it and some people regret using it, despite it being relatively popular.

There are some Rust things for doing VST (and AU) development. Here's one that I've seen a few things made with: https://github.com/robbert-vdh/nih-plug/tree/master I wouldn't worry too much about the differences between C++ and Rust in this world. Audio software tends to be buggy, so the bar for being considered 'good enough' is pretty low.

There’s nih-plug for VST and CLAP plugins

As it's written in Rust, perhaps it could be implemented as a CLAP plugin. This is a nice framework I've been playing with https://github.com/robbert-vdh/nih-plug/

The potential is both in terms of moving away from proprietary corporate-controlled standards (but also still providing shared wrappers to support those "legacy" :) audio plugin formats) and supporting Rust as a first class development language (via e.g. https://github.com/robbert-vdh/nih-plug).

Yep, Steinberg licensing sucks. I suggest you use nih-plug, it has a much nicer Rust API and supports generating both VST3 and CLAP plugins from your code. I'm only targetting CLAP nowadays, but unfortunately not many DAW's support it.

Your best bet is probably nih-plug.