WebCodecsOpusRecorder VS workerd

Take for example https://github.com/guest271314/WebCodecsOpusRecorder. There was no roadmap anywhere in the wild for how to write Opus encoded packets produced by WebCodecs AudioEncoder to a single file, including the capability to include media metadata such as artist, album, artwork in the file, for use with Media Session API - without a media container - and play back the file in the browser. So how would the documentation be automatically generated?

We can write a Uint32Array, JSON, ArrayBuffer adjacent to the other in a Blob. That means we can write our own algorithm for stroing arbitrary data and reading the data back. E.g., we can write the length of JSON containing configuration metadata, for example image artwork and offsets of ArrayBuffers before the JSON, then after the JSON write a series of ArraBuffers next to each other, then read the length of the JSON stored in the first 4 bytes as a Uint32Array, get the variable length of the JSON following the Uint32Array, read the offsets in an array in JSON configuration, then read each offset of ArrayBuffers stored in the file. Kind of how the Native Messaging protocol works, extended for the capability to write arbitrary data to a file with decoding instruction set encoded within the file itself, so we can, for example, write Opus encoded audio from WebCodecs, image artwork, artists, title, album data to a file, and then read the file, display images, artist, album data written therein in using Media Session API, and stream the audio using Medis Source Extensions, or decode the audio to a WAV file from Opus compression in the browser. E.g., https://github.com/guest271314/WebCodecsOpusRecorder. Bonus: The resulting file size, excluding the images serialized in the file, audio for audio, is less than Opus encoded in WebM file, the default container for MediaRecorder output on Chrome on Linux.

Not at all. Here I encoded Opus audio output by WebCodecs AudioEncoder, write the encoded chunks to a single file, preceded by JSON configuration and indexes of the discrete encoded chunks, optioally included media metadata such as artists, album, artwork, so we can fetch the first 4 bytes to read the Uint32Array at the beginning of the file to get the offsets information, then make separate range requests for the given timeslice(s) or media and playback that media https://github.com/guest271314/WebCodecsOpusRecorder.

As long as the encoder and decoder are on the same page, and you keep track of offsets, you can do whatever you want. Particularly using a Blob. Here https://github.com/guest271314/WebCodecsOpusRecorder/blob/main/WebCodecsOpusRecorder.js I write a Uint32Array, JSON, and ArrayBuffers containing WebCodecs Opus encoded audio, and optionally images and metadata for Media Session API to the same file, and play the file back in the browser, in pertinent part

Another fun project was encoding Opus packets output by WebCodecs AudioEncoder to a single file, and playing the file back in the browser https://github.com/guest271314/WebCodecsOpusRecorder. There was no road map to do that.

I never had an issue encoding and decoding Opus packets using the above approaches https://github.com/guest271314/WebCodecsOpusRecorder.

I carried on an developed a way to do just that, and save all packets to a single file and play back that file several ways. The resulting file winds up being more compact than Opus encoded ina WebM container. I then added a way to include images in the file to support Media Session metadata https://github.com/guest271314/WebCodecsOpusRecorder. Et al.

Recording raw Opus packets produced by WebCodecs AudioEncoder to a single file - without a media container such as Matroska, WebM, MP3, AAC, etc. - then playing back the file. You can test for yourself on Chrome or Chromium here https://guest271314.github.io/WebCodecsOpusRecorder/webcodecs-opus-recorder-mse-wav-player.html. Record your microphone or other device remapped as a microphone, save the file, then upload the file and play it back. I included the ability to also store an image in the file for media session metadata support, so we get to see same or similar image you see at global media controls when playing for example a YouTube video.

There was no roadmap for how to write Opus packets produced by Chrome's WebCodecs AudioEncoder to a single file - without writing the Opus packs to a media container such as Matroska or WebM. I just know it could be done, and used my experience testing Native Messaging to use the concept of preceding the data with a Uint32Array containing the length of the file, in this case, including the offsets of each packet to JSON array, then writing the algorith to extrack that data for playback https://github.com/guest271314/WebCodecsOpusRecorder.

This is how I write Opus packets to a file without a container and playback using Media Source Extension or as WAV file https://github.com/guest271314/WebCodecsOpusRecorder

Standards bodies only standardize things after they've been proven to work. You can't standardize a new idea before offering it to the market. It's hard enough to get just one vendor to experiment with an idea (it literally took me years to convince everyone inside Cloudflare that we should build Durable Objects). Getting N competing vendors to agree on it -- before anything has been proven in the market -- is simply not possible.

But the Durable Objects API is not complicated and there's nothing stopping competing platforms from building a compatible product if they want. Much of the implementation is open source, even. In fact, if you build an app on DO but decide you don't want to host it on Cloudflare, you can self-host it on workerd:

https://github.com/cloudflare/workerd

In any case, I welcome this initiative with my open hands and look forward all the cool apps that people will now build with this!

[1] https://pyodide.org/

[2] https://github.com/cloudflare/workerd/blob/main/docs/pyodide...

[3] https://github.com/cloudflare/workerd/pull/1875

For ref:

- https://blog.cloudflare.com/workerd-open-source-workers-runt...

- https://github.com/cloudflare/workerd

workerd

I think this is for people who want to run their own cloudflare workers (sort of) and since nobody wants to run full node for that, they want a small runtime that just executes js/wasm in an isolated way. But I wonder why they don't tell me how I can be sure that this is safe or how it's safe. Surely I can't just trust them and it explicitly mentions that it still has file IO so clearly there is still work I need to do customize the isolation further. But then they don't show any info on that core usecase. But then that's probably because they don't really want you to use this to run it on your own, they are selling you on running things on their edge platform called "Wasmer Edge". So that's probably why this is so light on information.. the motivation isn't to get you to use this yourself, just to use this their hosted edge platform. But then I wonder why I wouldn't just use https://github.com/cloudflare/workerd which is also open source. Surely that is fast enough? If not then it should show some benchmarks?

i love how the main reference for workerd can be just one capnp file.

https://github.com/cloudflare/workerd/blob/main/src/workerd/...

this changed my world how i think about computing on the web.

if there was just a good enough js library as for lua and you could directly send capnp messages to workerd instead of always going through files. I guess one day i have to relearn c++ and understand how the internals actually work.

A significant chunk of it is open source: https://github.com/cloudflare/workerd/

Some of the configuration files for applications wind up being an entire language unto themselves, e.g., https://github.com/cloudflare/workerd/blob/1b5057f2bfcfedf146f6f79ff04e99903d55412b/src/workerd/io/compatibility-date.capnp

Edge runtimes are very good alternatives to containers that shouldn't be dismissed for "not being containers". They're often faster, more scalable, and cheaper than containers. Them being so lightweight also enable a "nanoservice architecture" – being able to run every service on a single computer instead of running different services on different computers and having to deal with network latency and unreliability.