unboxed VS bytestring

With Lev as defined here.

Check out unboxed. I think that could be a good starting point. I recently updated it to the 9.2.1 alpha here: https://github.com/noughtmare/unboxed/commit/b6338e68540a2c62a6aae6ded0dcdc3b83106916. I should check again if the this issue is fixed in this rc.

I tried it out and it is really not usable yet unfortunately. There are several problems. I think the main issue is that you need two versions of all functions and data types. And currently classes like Num are not levity-polymorphic (there is a proposal about this), so you can't use + to add two "real" inductive peano naturals. One problem with making type classes levity-polymorphic is that top-level bindings can't be levity-polymorphic, so mempty cannot be levity-polymorphic. A workaround is to use a type class to levitate levity-polymorphic variables, but that has disadvantages too. Levitated values are not shared and must be recomputed at every call site.

The documentation is itself fairly funny, for those who don’t care to click ahead:

> This "function" has a superficial similarity to ‘unsafePerformIO’ but it is in fact a malevolent agent of chaos. It unpicks the seams of reality (and the IO monad) so that the normal rules no longer apply. It lulls you into thinking it is reasonable, but when you are not looking it stabs you in the back and aliases all of your mutable buffers. The carcass of many a seasoned Haskell programmer lie strewn at its feet.

> Witness the trail of destruction:

    https://github.com/haskell/bytestring/commit/71c4b438c675aa360c79d79acc9a491e7bbc26e7

If you bring in efficient strings from bytestring, densely packed arrays from vector, and an in-place sort from vector-algorithms, you can bring it down to 275ms (uses 19MB of mem).

bytestring

Note that this release is broken for Windows.

I've opened https://github.com/haskell/bytestring/issues/455 so the situation can be improved. You're very welcome to chime in on the discussion or to contribute some of the missing documentation yourself! :)

Highlights from the changelog:

Well, ByteString in particular should not have an IsString instance in a new report. That's pretty clear by https://github.com/haskell/bytestring/issues/140 : the concensus is that there is no good solution right now, but it should not have gotten an IsString instance in the first place. If a theoretical new Haskell Report 202x includes OverloadedStrings (as it should) to handle string literals analogously to numeric literals, I'd expect it to not give ByteString (which is really just a collection of octets) an IsString instance, with all it's issues and rattail due to the encoding question being implicitized.

Standard streaming libraries. They are being written by people that make the effort to understand performance and I have a hope that they make sure their streams run in linear space under any optimizations. It is curious and unsettling that we have standard lazy text and byte streams at the same time — and the default lazy lists, of course. I have been doing some work on byte streams and what I found out is that there is no way to check that your folds are actually space constant even if the value in question is a primitive, like say a byte — thunks may explode and then collapse over the run time of a single computation, defying any effort at inspection.