rust-base64 VS abseil-cpp

It would be interesting to compare your implementation and the most popular implementation for Rust+base64: https://github.com/marshallpierce/rust-base64

I am writing a CLI util in rust that utilizes xml-rpc-rs to talk to an rtorrent server and I would like to be able to add torrent files. OK according to the python implementation, which some of the rtorrent developers have said is good, of xmlrpc-client it uses this base64 format: https://datatracker.ietf.org/doc/html/rfc2045.html#section-6.8 I base64 encode /some/file/foo.torrent and send it up. OK!

Funny that you claim base64 forbids unsafe code while linking a PR where the current maintainer of the crate explicitly agrees that unsafe for the purpose of SIMD-acceleration is a-okay. Did you by any chance meant to link https://github.com/marshallpierce/rust-base64/pull/114 instead? ;)

> It does need to be in the standard library

When I say that something “has to be in the standard library”, I mean that it can’t be implemented outside the standard library. That’s certainly not the case here. You’re using an outright bad definition of “need” here—subjective opinion rather than objective requirement.

> because everyone needs it

This is factually wildly wrong. I wrote a fair bit more here but decided it wasn’t helpful. Précis: web stuff tends to load it indirectly (though amusingly most of the time actually not use it, so that Base64 code won’t actually end up in your binary), but it’s not terribly common outside of internet stuff to reach for Base64.

I’ll leave just one more remark about Base64: once things are in the standard library, breaking changes can no longer be made; the base64 crate is still experiencing breaking changes (<https://github.com/marshallpierce/rust-base64/blob/master/RE...>, 0.12 and 0.13 were last year and 0.20 is not released), largely for performance reasons.

Please don’t just call the thin-std approach “problematic” without acknowledging that the alternative is at least as problematic, just with a different set of caveats.

Many of these have their own tracking issues on the path to v1.0. For example see this one for base64.

I see base64. If the standard library has base64 encoding, go ahead and use it. But as a third-party dependency? Again, base64 encoding and decoding is trivial. I've written this a few times myself. It's not worth a dependency.

Yeah, it's nice! And Abseil does it, IFF you use LLVM libc++.

https://github.com/abseil/abseil-cpp/blob/master/absl/string...

The standard adopted it as resize_and_overwrite. Which I think is a little clunky.

This may be confusing to those familiar with Google's libraries. The baseline is the Go BTree, which I personally never heard of until just now, not the C++ absl::btree_set. The benchmarks aren't directly comparable, but the C++ version also comes with good microbenchmark coverage.

https://github.com/google/btree

https://github.com/abseil/abseil-cpp/blob/master/absl/contai...

An implementation of call_once that accommodates callbacks that throw: https://github.com/abseil/abseil-cpp/blob/master/absl/base/c...

I wouldn't say it's that cryptic. It's just a few bitwise rotations/shifts/xor operations.

You can see hashing optimizations as well https://www.deepmind.com/blog/alphadev-discovers-faster-sort..., https://github.com/abseil/abseil-cpp/commit/74eee2aff683cc7d...

I was one of the members who reviewed expertly what has been done both in sorting and hashing. Overall it's more about assembly, finding missed compiler optimizations and balancing between correctness and distribution (in hashing in particular).

It was not revolutionary in a sense it hasn't found completely new approaches but converged to something incomprehensible for humans but relatively good for performance which proves the point that optimal programs are very inhuman.

Note that for instructions in sorting, removing them does not always lead to better performance, for example, instructions can run in parallel and the effect can be less profound. Benchmarks can lie and compiler could do something differently when recompiling the sort3 function which was changed. There was some evidence that the effect can come from the other side.

For hashing it was even funnier, very small strings up to 64 bit already used 3 instructions like add some constant -> multiply 64x64 -> xor upper/lower. For bigger ones the question becomes more complicated, that's why 9-16 was a better spot and it simplified from 2 multiplications to just one and a rotation. Distribution on real workloads was good, it almost passed smhasher and we decided it was good enough to try out in prod. We did not rollback as you can see from abseil :)

But even given all that, it was fascinating to watch how this system was searching and was able to find particular programs can be further simplified. Kudos to everyone involved, it's a great incremental change that can bring more results in the future.

Check out https://abseil.io. It offers absl::optional, which is a backport of std::optional.