Replacing WebRTC: real-time latency with WebTransport and WebCodecs

• web-codecs

  14 899 8.1 HTML

  WebCodecs is a flexible web API for encoding and decoding audio and video.

  

https://blog.paul.cx/post/audio-video-synchronization-with-t... has some background, https://github.com/w3c/webcodecs/blob/main/samples/lib/web_a... is part of a full example that you can run using web codecs, web audio, audioworklet SharedArrayBuffer, and does A/V sync.

If it doesn't answer your question let me know because I wrote both (and part of the web audio spec, and part of the webcodecs spec).

• webrtc-rtptransport

  1 5 7.4 Bikeshed

  Repository for the RTPTransport specification of the WebRTC Working Group

  

That's so exciting! I had no idea you were working on this :)

Here [0] is a link for anyone that is looking for it.

[0] https://github.com/w3c/webrtc-rtptransport/blob/main/explain...

• WorkOS

  workos.com sponsored

  The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.

  

• webrtc-for-the-curious

  13 1,879 5.2 Python

  WebRTC for the Curious: Go beyond the APIs

  

For the WebRTC jargon check out https://webrtcforthecurious.com/

If that still doesn’t cover enough I would love to hear! Always trying to make it better.

• moq-js

  2 97 8.6 TypeScript

  Typescript library for Media over QUIC

I'm using AudioWorklet and SharedArrayBuffer. Here's my code: https://github.com/kixelated/moq-js/tree/main/lib/playback/w...

It's just a lot of work to get everything right. It's kind of working, but I removed synchronization because the signaling between the WebWorker and AudioWorklet got too convoluted. It all makes sense; I just wish there was an easier way to emit audio.

While you're here, how difficult would it be to implement echo cancellation? The current demo is uni-directional but we'll need to make it bi-directional for conferencing.

• WebKit

  151 7,438 10.0

  Home of the WebKit project, the browser engine used by Safari, Mail, App Store and many other applications on macOS, iOS and Linux.

  

It's being worked on now: https://github.com/WebKit/WebKit/pull/17320

• libav.js

  2 267 9.3 JavaScript

  This is a compilation of the libraries associated with handling audio and video in ffmpeg—libavformat, libavcodec, libavfilter, libavutil, libswresample, and libswscale—for emscripten, and thus the web.

I'm currently working with WebCodecs to get (the long awaited) frame-by-frame seeking and reverse playback working in the browser. And it even seems to work, albeit the VideoDecoder queuing logic seems to give some grief for this. Any tips on figuring out how many chunks have to be queued for a specific VideoFrame to pop out?

An aside: to work with video/container files, be sure to check the libav.js project that can be used to demux streams (WebCodecs don't do this) and even used as a polyfill decoder for browsers without WebCodec support!

https://github.com/Yahweasel/libav.js/

Suggest a related project