WebCodecsOpusRecorder VS WebRTC

Take for example https://github.com/guest271314/WebCodecsOpusRecorder. There was no roadmap anywhere in the wild for how to write Opus encoded packets produced by WebCodecs AudioEncoder to a single file, including the capability to include media metadata such as artist, album, artwork in the file, for use with Media Session API - without a media container - and play back the file in the browser. So how would the documentation be automatically generated?

We can write a Uint32Array, JSON, ArrayBuffer adjacent to the other in a Blob. That means we can write our own algorithm for stroing arbitrary data and reading the data back. E.g., we can write the length of JSON containing configuration metadata, for example image artwork and offsets of ArrayBuffers before the JSON, then after the JSON write a series of ArraBuffers next to each other, then read the length of the JSON stored in the first 4 bytes as a Uint32Array, get the variable length of the JSON following the Uint32Array, read the offsets in an array in JSON configuration, then read each offset of ArrayBuffers stored in the file. Kind of how the Native Messaging protocol works, extended for the capability to write arbitrary data to a file with decoding instruction set encoded within the file itself, so we can, for example, write Opus encoded audio from WebCodecs, image artwork, artists, title, album data to a file, and then read the file, display images, artist, album data written therein in using Media Session API, and stream the audio using Medis Source Extensions, or decode the audio to a WAV file from Opus compression in the browser. E.g., https://github.com/guest271314/WebCodecsOpusRecorder. Bonus: The resulting file size, excluding the images serialized in the file, audio for audio, is less than Opus encoded in WebM file, the default container for MediaRecorder output on Chrome on Linux.

Not at all. Here I encoded Opus audio output by WebCodecs AudioEncoder, write the encoded chunks to a single file, preceded by JSON configuration and indexes of the discrete encoded chunks, optioally included media metadata such as artists, album, artwork, so we can fetch the first 4 bytes to read the Uint32Array at the beginning of the file to get the offsets information, then make separate range requests for the given timeslice(s) or media and playback that media https://github.com/guest271314/WebCodecsOpusRecorder.

As long as the encoder and decoder are on the same page, and you keep track of offsets, you can do whatever you want. Particularly using a Blob. Here https://github.com/guest271314/WebCodecsOpusRecorder/blob/main/WebCodecsOpusRecorder.js I write a Uint32Array, JSON, and ArrayBuffers containing WebCodecs Opus encoded audio, and optionally images and metadata for Media Session API to the same file, and play the file back in the browser, in pertinent part

Another fun project was encoding Opus packets output by WebCodecs AudioEncoder to a single file, and playing the file back in the browser https://github.com/guest271314/WebCodecsOpusRecorder. There was no road map to do that.

I never had an issue encoding and decoding Opus packets using the above approaches https://github.com/guest271314/WebCodecsOpusRecorder.

I carried on an developed a way to do just that, and save all packets to a single file and play back that file several ways. The resulting file winds up being more compact than Opus encoded ina WebM container. I then added a way to include images in the file to support Media Session metadata https://github.com/guest271314/WebCodecsOpusRecorder. Et al.

Recording raw Opus packets produced by WebCodecs AudioEncoder to a single file - without a media container such as Matroska, WebM, MP3, AAC, etc. - then playing back the file. You can test for yourself on Chrome or Chromium here https://guest271314.github.io/WebCodecsOpusRecorder/webcodecs-opus-recorder-mse-wav-player.html. Record your microphone or other device remapped as a microphone, save the file, then upload the file and play it back. I included the ability to also store an image in the file for media session metadata support, so we get to see same or similar image you see at global media controls when playing for example a YouTube video.

There was no roadmap for how to write Opus packets produced by Chrome's WebCodecs AudioEncoder to a single file - without writing the Opus packs to a media container such as Matroska or WebM. I just know it could be done, and used my experience testing Native Messaging to use the concept of preceding the data with a Uint32Array containing the length of the file, in this case, including the offsets of each packet to JSON array, then writing the algorith to extrack that data for playback https://github.com/guest271314/WebCodecsOpusRecorder.

This is how I write Opus packets to a file without a container and playback using Media Source Extension or as WAV file https://github.com/guest271314/WebCodecsOpusRecorder

It's worth a try though to push back. We are not talking holding off a bunch of murderous Russian troops. It's not a good look trying to recall " ... and they came for me" either.

Just push back. Here's an example: Reolink are a Chinese company, what makes cameras - nothing intrinsically wrong with that but you should expect them to be be required to comply with any requirements the CCP might ... require. Reolink are also quite a savvy bunch and have gradually ensured that their products don't actually require an internet connection, at all. They do offer an app and the requirements of using the app are that the cams need to see the interwebs and be gatewayed by systems that are eventually subject to the CCP.

Now this isn't quite yet perfect. Rio cams have offered ONVIF for at least five years, so Zoneminder, Frigate and all the rest can be your NVR. The camera's VLAN can be firewalled off from the internet. Mine is called THINGS and it is net door to SEWER for stuff I really worry about!

Their doorbell offering is pretty decent but two way comms needs some handling. At the moment their app is the best bet for functionality but there are signs that Home Assistant with webrtc - https://github.com/AlexxIT/WebRTC should be OK.

It is not impossible to live without prop software. At least care and try.

For Home Assistant, I found WebRTC to work pretty well, particularly on substreams, but honestly I just ended up using iframes of Blue Iris instead as I found that to be more reliable and more versatile (I can easily access stored videos from the tablet, etc)

I stream live cams using this https://github.com/AlexxIT/WebRTC

Not sure if exactly the same issue, but I was having big problems with RTSP framerate and latency (about 2seconds) - and solved by using this WebRTC addon, which works super well: https://github.com/AlexxIT/WebRTC