Parallel streaming in Haskell: Part 1 – Fast, efficient, and fun

• attoparsec-conduit

  2 883 3.8 Haskell

  A streaming data library

> makes me think that there aren't any conduits that have more data than can fit in memory on a single machine

Just because you can dedup a conduit doesn't mean you have to. We use conduits for streaming gigabytes-large files while staying within megabytes of memory use – ensuring that is one of the main selling points of libraries like conduit: https://github.com/snoyberg/conduit#readme

• FS2

  17 2,325 9.5 Scala

  Compositional, streaming I/O library for Scala

  

Thanks for the explanation!

So it's pull based and not push based like most other streams lib.

Does maybe someone know how this compares to FS2 or Iteratees than? (Both are also pull based streaming solutions).

https://fs2.io/#/

https://en.wikipedia.org/wiki/Iteratee

Looks quite similar to me. Is the Scala FS2 lib maybe even a clones of the Haskell solution? Or are they different in important aspects?

• InfluxDB

  www.influxdata.com sponsored

  Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

  

Suggest a related project