nodejs-polars VS streams

Polars: A blazingly fast DataFrame library written in Rust for data manipulation and analysis

StarlingMonkey is a JavaScript runtime we’ve built together with our friends at Fastly and contributed to the Bytecode Alliance. It’s built on top of SpiderMonkey in a highly modular way, making it easy to configure as needed for our use case. Crucially, it comes with an implementation of key web APIs that substantially improve compatibility with the web ecosystem, like the fetch API for handling outgoing HTTP requests, key parts of the Service Workers spec for handling incoming requests, streaming processing of request and response bodies using the web’s Streams API streamssetTimeout, and setInterval.

Every single JS future is boxed. Moreover, they aren't just boxed, they are often backed by a hashmap (which may or may not be optimised away by the JIT). Elaborate allocation-free async is not an apple-to-apples comparison, that's my point.

JS does support concurrent execution, Promise.all is an example. Without it, JS async would make little sense. The problem very much exists there, and try-catch is only a surface-level answer. As you can see here [1], the interaction of cancellation and async in JS is at least just as (or more) complex than in Rust.

By the way, multithreading has little to do with Pin. I presume you're thinking of Send bounds.

"To work at all" is very dismissive. It's complex, but very well abstracted, well defined, and robust, that complexity is essential. Again, look at [1], JS async is hardly less complex, but also much more vague and ill-defined.

[1]: https://github.com/whatwg/streams/issues/1255

Yup, this is what WHATWG's Streams spec[0] (linked in the article) says. It defines backpressure as a "process of normalizing flow from the original source according to how fast the chain can process chunks" where the reader "propagates a signal backwards through the pipe chain".

Mozilla's documentation[1] similarly defines backpressure as "the process by which a single stream or a pipe chain regulates the speed of reading/writing".

The article confuses backpressure (the signal used for regulation of the flow) with the reason backpressure is needed (producers and consumers working at different speeds). It should be fairly clear from the metaphor, I would have thought: With a pipe of unbounded size there is no pressure. The pressure builds up when consumer is slower than producer, which in turn slows down the producer. (Or the pipe explodes, or springs a leak and has to drop data on the ground.)

[0] https://streams.spec.whatwg.org/#pipe-chains

[1] https://developer.mozilla.org/en-US/docs/Web/API/Streams_API...

// https://streams.spec.whatwg.org/#example-transform-identity const { writable, readable } = new TransformStream(); fetch("...", { body: readable }).then(response => /* ... */); const writer = writable.getWriter(); writer.write(new Uint8Array([0x73, 0x74, 0x72, 0x65, 0x61, 0x6D, 0x73, 0x21])); // "streams!" writer.close();

// AudioWorkletStream // Stream audio from Worker to AudioWorklet // guest271314 2-24-2020 let port; onmessage = async e => { 'use strict'; if (!port) { [port] = e.ports; port.onmessage = event => postMessage(event.data); } const { urls } = e.data; // https://github.com/whatwg/streams/blob/master/transferable-streams-explainer.md const { readable, writable } = new TransformStream(); (async _ => { for await (const _ of (async function* stream() { while (urls.length) { yield (await fetch(urls.shift(), {cache: 'no-store'})).body.pipeTo(writable, { preventClose: !!urls.length, }); } })()); })(); port.postMessage( { readable, }, [readable] ); };