txiki.js VS AudioWorkletStream

txiki.js

// https://github.com/denoland/deno/discussions/17236#discussioncomment-4566134 // https://github.com/saghul/txiki.js/blob/master/src/js/core/tjs/eval-stdin.js async function readFullAsync(length, buffer = new Uint8Array(65536)) { const data = []; while (data.length < length) { const input = await open("/dev/stdin"); let { bytesRead } = await input.read({ buffer }); await input.close(); if (bytesRead === 0) { break; } data.push(...buffer.subarray(0, bytesRead)); } return new Uint8Array(data); }

I've only dabbled with the shared libraries once, with the help of the owner of the lnked repository. I wound up solving the issue adjusting the code I was working on https://github.com/saghul/txiki.js/issues/294.

I have written the same code in C++ https://github.com/guest271314/captureSystemAudio/blob/master/native_messaging/capture_system_audio/capture_system_audio.cpp, Python https://github.com/guest271314/captureSystemAudio/blob/master/native_messaging/capture_system_audio/capture_system_audio.py, and QuickJS (C source code) https://github.com/saghul/txiki.js/issues/294. QuickJS is the only implementation that winds up clipping the end of real-time audio capture.

QuickJS is awesome, simple to use and very easy to integrate.

Over the past few years I slowly built a small JS runtime using QuickJS as the engine and libuv as the platform layer, amongst other things, in case anyone wants to take a look: https://github.com/saghul/txiki.js

// Exits half way through reading response when --max-old-space-size=6 is set // Exits immediately when --jitless flag is set // // Usage: // // port.postMessage({ // url: 'https://github.com/guest271314/AudioWorkletStream/raw/master/house--64kbs-0-wav', // method: 'get', // body: null // })

Yes. Fetching a single or multiple files for an infinite stream of audio https://github.com/guest271314/AudioWorkletStream/blob/master/worker.js. Streaming (real-time) audio is non-trivial. Any gaps or glitches in playback will be audible to the user. We could test for expected Float32Arrays. I would suggest complentary manual test in, e.g., WPT to determine audio output does not have gaps or glitches; and renders the expected playback rate.

One way to do that is utilizing Native Messaging on Chromium or Firefox https://github.com/guest271314/native-messaging-espeak-ng, and https://github.com/guest271314/webtransport/blob/main/webTransportEspeakNg.js for some WebTransport experiments. Technically we don't need a local server. We can stream and parse the WAV directly and pipe to AudioWorklet or a MediaStreamTrackGenerator https://github.com/guest271314/AudioWorkletStream. The same is true for speech recognition, where audio is piped to the local application and text or JSON piped back. Note also espeak-ng has been compiled to WebAssembly. I created native-messaging-espeak-ng for the ability to pass SSML directly to espeak-ng.

You can stream audio and/or video over HTTP using fetch() https://github.com/guest271314/AudioWorkletStream as long as you know how to parse the codec, if the media is encoded.

Re using a dedicated Worker and AudioWorklet to stream, see, e.g., https://github.com/guest271314/AudioWorkletStream; https://plnkr.co/edit/nECtUZ.