portable-simd VS rust-base64

Others have already addressed the "unsafe" smell.

I think the bigger point here is that doing SIMD in Rust is still painful.

There are efforts like portable-simd [1] to make this better, but in practice, many people are dropping down to low-level SIMD intrinsics and/or inline assembly, which are no better than their C equivalents.

[1]: https://github.com/rust-lang/portable-simd

Jubilee and Caleb Zulawski for their tireless work on the portable SIMD project. It will land, some day, and when it does it's going to be an amazing boon for the project.

The portable SIMD project implements an as_simd() function for slices. The basics are that you get 3 slices and the middle one is a SIMD slice. It allows for fast aligned loads of the data, which could matter if your algorithm is becoming memory bound; it is also a convenient and safe abstraction. In my case, I actually have 2 vectors (say, x and y). I can take them apart using as_simd() like so:

Rust has generics and monomorphization. You can write the algorithm once and compile for multiple targets. rust-lang/portable-simd

This Rust issue [0] was the best short summary of what an SIMD Shuffle is I could find:

„A "shuffle", in SIMD terms, takes a SIMD vector (or possibly two vectors) and a pattern of source lane indexes (usually as an immediate), and then produces a new SIMD vector where the output is the source lane values in the pattern given.“

[0] https://github.com/rust-lang/portable-simd/issues/11

Yet by default it generates code which is only compatible with Pentium4 or newer. In fact lots of serious issues relate to older CPUs and rustc developers plan is to declare them closed when they would be able to drop i686 support (all AMD CPUs which support SSE2 support x86-64, too while Intel situation is mess).

The standard API crate for portable simd is at https://github.com/rust-lang/portable-simd, but using this requires nightly, which I don't want to do. I'd like to use a crate for simd that works on both x86_64 and wasm in stable rust. wide looks fine for this purpose. Are there any potentially better choices?

See: https://github.com/rust-lang/portable-simd

It would be interesting to compare your implementation and the most popular implementation for Rust+base64: https://github.com/marshallpierce/rust-base64

I am writing a CLI util in rust that utilizes xml-rpc-rs to talk to an rtorrent server and I would like to be able to add torrent files. OK according to the python implementation, which some of the rtorrent developers have said is good, of xmlrpc-client it uses this base64 format: https://datatracker.ietf.org/doc/html/rfc2045.html#section-6.8 I base64 encode /some/file/foo.torrent and send it up. OK!

Funny that you claim base64 forbids unsafe code while linking a PR where the current maintainer of the crate explicitly agrees that unsafe for the purpose of SIMD-acceleration is a-okay. Did you by any chance meant to link https://github.com/marshallpierce/rust-base64/pull/114 instead? ;)

> It does need to be in the standard library

When I say that something “has to be in the standard library”, I mean that it can’t be implemented outside the standard library. That’s certainly not the case here. You’re using an outright bad definition of “need” here—subjective opinion rather than objective requirement.

> because everyone needs it

This is factually wildly wrong. I wrote a fair bit more here but decided it wasn’t helpful. Précis: web stuff tends to load it indirectly (though amusingly most of the time actually not use it, so that Base64 code won’t actually end up in your binary), but it’s not terribly common outside of internet stuff to reach for Base64.

I’ll leave just one more remark about Base64: once things are in the standard library, breaking changes can no longer be made; the base64 crate is still experiencing breaking changes (<https://github.com/marshallpierce/rust-base64/blob/master/RE...>, 0.12 and 0.13 were last year and 0.20 is not released), largely for performance reasons.

Please don’t just call the thin-std approach “problematic” without acknowledging that the alternative is at least as problematic, just with a different set of caveats.

Many of these have their own tracking issues on the path to v1.0. For example see this one for base64.

I see base64. If the standard library has base64 encoding, go ahead and use it. But as a third-party dependency? Again, base64 encoding and decoding is trivial. I've written this a few times myself. It's not worth a dependency.