attoparsec-conduit VS wai-conduit

Working through Learn You A Haskell is a good start.

After that, I honestly think you'll get the best bang-for-buck by reading library-specific tutorials. If you play with enough of the libraries the rest of the language more or less falls into place.

Conduit is a pretty ok streaming library, and has good documentation: https://github.com/snoyberg/conduit#readme

Lens gives you a lot of useful features that more or less correspond to stuff like Getters and Setters in something like Java, and the tutorials for it get into some helpful details about writing Haskell code: https://hackage.haskell.org/package/lens

Otherwise it's basically a lot of "just build shit, and don't be afraid to feel confused" and it'll fall into place.

> makes me think that there aren't any conduits that have more data than can fit in memory on a single machine

Just because you can dedup a conduit doesn't mean you have to. We use conduits for streaming gigabytes-large files while staying within megabytes of memory use – ensuring that is one of the main selling points of libraries like conduit: https://github.com/snoyberg/conduit#readme

found some context https://github.com/yesodweb/wai/pull/931

Yes. Not because of the developer, but because of how extremely flexible and dynamic the Lisp-family languages are. The power and joy of Lisp is in how it's almost a meta-language, so every project can become its own EDSL. The most famous (infamous?) example of this is Vacietis[2], which is a Common Lisp library that allows C code to be imported directly(!!).

[0] IIRC the Yesod framework's Warp does well on benchmarks, and when you look at code like https://github.com/yesodweb/wai/blob/master/warp/Network/Wai... you can see the lengths they had to go through to work around the choice of implementation language.

[1] Go has a garbage collector, but exposes the stack/heap distinction more directly than Haskell, so it's easier to write allocation-free code in hot paths.

[2] https://github.com/vsedach/Vacietis

My $3/mo vultr box can handle HN loads easily when using a fast backend (I've settled on https://github.com/yesodweb/wai based apps - the only thing that has worked well for me so far).

What kind of metrics do you derive "ton of stuff" from? It seems like the largest blocker is Cryptonite. It's unreasonable to let a handful of packages keep back Nightly. You can now run Warp without it. How does your list of essential blockers for 9.2 look like?

No, Wai is the common abstraction, sort of like connect. Warp is one (only AFAIK) server implementation of it. Both Wai and Warp are developed along side with Yesod, you can find their source code here https://github.com/yesodweb/wai

WAI