free-arrow VS Reactive Streams

my first thought was something something dependent types (Idris, Agda), but it also sounds like TS-like structural typing with a Rust-like Result type. proving that every incoming message is either parsed correctly or we return an error seems to be the basic building block. and then every transformation should be other pure functions.

thought I guess you mean something more top-downish? for that there's "program interpretation" ( https://github.com/AdrielC/free-arrow )

plus something very heavy-handed https://deepai.org/publication/a-coq-based-synthesis-of-scal...

Interesting post, although the use of `cats.IO` feels a little shoehorned. I've found some nice cats/akka synergy by abstracting over flows using a `cats.arrow.Arrow` instance for `akka.stream.scaladsl.Flow`. That might be just as shoehorned though, since I haven't yet worked out whether akkas `Flow` forms a lawful Arrow

Taken from https://www.reactive-streams.org/

Reactive programming is a paradigm that focuses on non-blocking and asynchronous processing of tasks. One set of specifications/abstractions for reactive programming on JVM is called Reactive Streams. Project Reactor is a message-driven, type-safe and functional implementation of Reactive Streams, and it is used by Spring (via spring-webflux module) to enable reactive web applications. Reactive streams model the data processing as a stream with one end producing the values and one end consuming them.

The reactive streams API provides the specification for non-blocking async streams processing with back pressure mechanism, and Project Reactor is an implementation written in java.

Cats Effect is a little more than just an IO effect implementation as they also provide an interface (or a standard) against implemented as typeclasses. You could think of it as a Java's Reactive Streams library which allows switching underlying implementation of actual effectful streaming.

NOTE: The Reactive Streams API and the implementation of it as provided by Spring WebFlux/Project Reactor is beyond the scope of this particular article. Please consult the appropriate documentation at the 'Web on Reactive Stack' Spring documentation site, any of several sessions I've delivered available on my YouTube channel, or by visiting the Reactive Streams and Project Reactor sites.

There is a very big design space for "Stream" APIs.

Microsoft's LINQ for instance can compile a stream operation into a SQL statement and JooQ does the same. That system offers query optimization and efficient joins that depend on the query system having complete visibility into the queries. indexes built ahead of time, etc.

Another extreme is a system like

https://www.reactive-streams.org/

that are especially good for apply a filter and map and other operations to a stream of real time events, e.g. instead of having a pull operation such as a for-loop over an Iterable, items go into the system from a stream.

I've worked on systems that use the later kind of streaming to run batch jobs and you can get great performance (780% speedup with 8 cpus) on crazy heterogenous workloads. You do have to be careful though to shut the system down or flush it out or otherwise you get wrong answers. Frequently those frameworks don't shut themselves down properly unless you implement clean shutdown yourself.

The point is that operators like "filter" and "map" and the rest are so powerful because they are portable between the minimal pidove up to a Hadoop cluster.

Backpressure is a protocol defining how to send a logical signal UP the stream with information about the downstream consumers readiness to receive more data. That backpressure signal follows the same path as data moving downstream, but in reverse. If downstream is not ready to consume, then upstream does does not send.

Reactive libraries like reactor are build on the Reactive Streams specification, just read that first sentence.

Not a well written article. "Fiber" was dropped by spec team way back in favor of "virtual thread". Mentions "Rx Java" but not http://www.reactive-streams.org/ as a standard for existing async IO. I mean anyone who has done reactive java long enough can tell you about various implementations! I expected a better article from infoworld.com