wai-conduit VS bytestring

found some context https://github.com/yesodweb/wai/pull/931

Yes. Not because of the developer, but because of how extremely flexible and dynamic the Lisp-family languages are. The power and joy of Lisp is in how it's almost a meta-language, so every project can become its own EDSL. The most famous (infamous?) example of this is Vacietis[2], which is a Common Lisp library that allows C code to be imported directly(!!).

[0] IIRC the Yesod framework's Warp does well on benchmarks, and when you look at code like https://github.com/yesodweb/wai/blob/master/warp/Network/Wai... you can see the lengths they had to go through to work around the choice of implementation language.

[1] Go has a garbage collector, but exposes the stack/heap distinction more directly than Haskell, so it's easier to write allocation-free code in hot paths.

[2] https://github.com/vsedach/Vacietis

My $3/mo vultr box can handle HN loads easily when using a fast backend (I've settled on https://github.com/yesodweb/wai based apps - the only thing that has worked well for me so far).

What kind of metrics do you derive "ton of stuff" from? It seems like the largest blocker is Cryptonite. It's unreasonable to let a handful of packages keep back Nightly. You can now run Warp without it. How does your list of essential blockers for 9.2 look like?

No, Wai is the common abstraction, sort of like connect. Warp is one (only AFAIK) server implementation of it. Both Wai and Warp are developed along side with Yesod, you can find their source code here https://github.com/yesodweb/wai

WAI

The documentation is itself fairly funny, for those who don’t care to click ahead:

> This "function" has a superficial similarity to ‘unsafePerformIO’ but it is in fact a malevolent agent of chaos. It unpicks the seams of reality (and the IO monad) so that the normal rules no longer apply. It lulls you into thinking it is reasonable, but when you are not looking it stabs you in the back and aliases all of your mutable buffers. The carcass of many a seasoned Haskell programmer lie strewn at its feet.

> Witness the trail of destruction:

    https://github.com/haskell/bytestring/commit/71c4b438c675aa360c79d79acc9a491e7bbc26e7

If you bring in efficient strings from bytestring, densely packed arrays from vector, and an in-place sort from vector-algorithms, you can bring it down to 275ms (uses 19MB of mem).

bytestring

Note that this release is broken for Windows.

I've opened https://github.com/haskell/bytestring/issues/455 so the situation can be improved. You're very welcome to chime in on the discussion or to contribute some of the missing documentation yourself! :)

Highlights from the changelog:

Well, ByteString in particular should not have an IsString instance in a new report. That's pretty clear by https://github.com/haskell/bytestring/issues/140 : the concensus is that there is no good solution right now, but it should not have gotten an IsString instance in the first place. If a theoretical new Haskell Report 202x includes OverloadedStrings (as it should) to handle string literals analogously to numeric literals, I'd expect it to not give ByteString (which is really just a collection of octets) an IsString instance, with all it's issues and rattail due to the encoding question being implicitized.

Standard streaming libraries. They are being written by people that make the effort to understand performance and I have a hope that they make sure their streams run in linear space under any optimizations. It is curious and unsettling that we have standard lazy text and byte streams at the same time — and the default lazy lists, of course. I have been doing some work on byte streams and what I found out is that there is no way to check that your folds are actually space constant even if the value in question is a primitive, like say a byte — thunks may explode and then collapse over the run time of a single computation, defying any effort at inspection.