web-codecs VS webrtc

Encoding alpha, please! https://github.com/w3c/webcodecs/issues/672

From developer.mozilla.org [0]

> WebCodecs API

> The WebCodecs API gives web developers low-level access to the individual frames of a video stream and chunks of audio. It is useful for web applications that require full control over the way media is processed. For example, video or audio editors, and video conferencing.

And from w3c [1]:

> The WebCodecs API allows web applications to encode and decode audio and video

All this looks really promising, I wouldn't have thought that we could use browsers directly to render videos. Maybe Puppeteer could then stream the content of the page it is rendering, for example a three.js animation.

[0] https://developer.mozilla.org/en-US/docs/Web/API/WebCodecs_A...

[1] https://github.com/w3c/webcodecs

As I understand it, Web Codecs is quite a ways away from being a web standard - it is currently just a draft[1] for a recommendation for a possible future standard.

Just beware of any major API changes or any indication that it might be dropped in its entirety!

[1]:https://w3c.github.io/webcodecs/

WebCodecs has Mozilla and Microsoft coauthors: https://w3c.github.io/webcodecs/

WebGPU was chartered in 2017, with more Apple people in the initial membership than Mozilla or Google people: https://gpuweb.github.io/admin/cg-charter.html

Google specifically has approached Apple for input about scheduler.postTask(), and they say they've presented the proposal before at a working group that Apple is part of: https://lists.webkit.org/pipermail/webkit-dev/2021-June/0319...

I won't disagree that there's a ton that can be improved about the process, but it seems pretty unsubstantiated to claim that Google is "rushing forwards as fast as possible" given that they've been spending years working on these APIs in public. You can see from the rest of the post that they're running some experiments which they're specifically choosing not to enable by default, though they could.

I work for PaperlessPost.com, and for our Flyer product we export mp4s from ffmpeg compiled to WASM, which is similar to ffmpeg.js but optimized for our use case which sounds similar. We have this WebAssembly method working well but iOS 15 (and the other major browsers) now support MediaExporter which might be a better way to go if you have something else to convert these files. This becomes and issue because you can't control the format MediaRecorder is recording to but the management of memory, the image quality, the compression, the performance hit will be more ideal than these other methods. OMGgif is very slow and will produce large files or very bad looking ones. Keep in mind that GIFs limit the colors so something that looks nice on the screen might not look the same after it is saved. The other thing to look out for is the WebCodecs APIs which should be the ultimate way to handle all of this in the future but it is only working in Chrome I think https://github.com/w3c/webcodecs

While in theory it might be possible to run ffmpeg through emscripten and run it in a web worker (I certainly assume someone has done this), it's not necessarily practical to try and use the same technique from node to transcode audio on the web. The good news is that the w3c has chartered a working group to focus on web codecs. While this is at the time of writing still in early stages, the working group is powering ahead on designing and proposing an API to enable media transcoding on the web, and hopefully that will become available to us in the near future.

I am really excited about https://github.com/sepfy/libpeer. It has examples ready for ESP32 etc....

When working on KVS I wasn't familiar with the embedded space at all. I saw 'heavyweight' embedded where you were running on Linux. Then you had RTOS/No OS at all. I wasn't prepared for these devices at all. If we can make WebRTC work in the embedded space I think it will really accelerate what developers are able to build!

Remotely driven cars, security cameras, robots in hospitals that bring iPads to infectious patients etc... Creative people are building amazing things. The WebRTC/video space needs to work harder and support them :)

-----

I love how diverse the WebRTC space is now. Outside of this implementation you have plenty of other options!

* https://github.com/shinyoshiaki/werift-webrtc (Typescript)

* https://github.com/pion/webrtc (Golang)

* https://github.com/webrtc-rs/webrtc (Rust)

* https://github.com/algesten/str0m (Rust)

* hhttps://github.com/sepfy/libpeer (C/Embedded)

* https://webrtc.googlesource.com/src/ (C++)

* https://github.com/sipsorcery-org/sipsorcery (C#)

* https://github.com/paullouisageneau/libdatachannel (C++)

* https://github.com/elixir-webrtc (Elixir)

* https://github.com/aiortc/aiortc (Python)

* GStreamer’s webrtcbin (C)

See https://github.com/sipsorcery/webrtc-echoes for examples of some running against each other.

This is not true, there are actually multiple WebRTC implementations in different languages besides the reference library: aiortc (python), libdatachannel (C++), sipsorcery (C#),webrtc-rs (rust), werift (Typescript), and Amazon Kinesis (C)

I am trying to get WebRTC working using Rust https://github.com/webrtc-rs/webrtc

I have opened an issue on GitHub [1], we can continue there.

[1] https://github.com/webrtc-rs/webrtc/issues/426

webrtc

webrtc.rs is a port of Pion (which we also use). It's a better fit for server-side use

I want to use peer-to-peer communication and data transfer for my next side project (client-server web app). I've been doing some research and WebRTC seems to be the only option for this on the client. There are a ton of libraries and product offering for facilitating STUN/TURN servers as a service, but I'm quite interested in learning more about these protocols. That being said, I'm not the best rust programmer (part of the reason of using Rust as the server is so that I can learn more), and the signalling protocols seem rather complicated. I've looked at https://github.com/webrtc-rs/webrtc and it seems promising.