rust-base64 VS semver-trick

It would be interesting to compare your implementation and the most popular implementation for Rust+base64: https://github.com/marshallpierce/rust-base64

I am writing a CLI util in rust that utilizes xml-rpc-rs to talk to an rtorrent server and I would like to be able to add torrent files. OK according to the python implementation, which some of the rtorrent developers have said is good, of xmlrpc-client it uses this base64 format: https://datatracker.ietf.org/doc/html/rfc2045.html#section-6.8 I base64 encode /some/file/foo.torrent and send it up. OK!

Funny that you claim base64 forbids unsafe code while linking a PR where the current maintainer of the crate explicitly agrees that unsafe for the purpose of SIMD-acceleration is a-okay. Did you by any chance meant to link https://github.com/marshallpierce/rust-base64/pull/114 instead? ;)

> It does need to be in the standard library

When I say that something “has to be in the standard library”, I mean that it can’t be implemented outside the standard library. That’s certainly not the case here. You’re using an outright bad definition of “need” here—subjective opinion rather than objective requirement.

> because everyone needs it

This is factually wildly wrong. I wrote a fair bit more here but decided it wasn’t helpful. Précis: web stuff tends to load it indirectly (though amusingly most of the time actually not use it, so that Base64 code won’t actually end up in your binary), but it’s not terribly common outside of internet stuff to reach for Base64.

I’ll leave just one more remark about Base64: once things are in the standard library, breaking changes can no longer be made; the base64 crate is still experiencing breaking changes (<https://github.com/marshallpierce/rust-base64/blob/master/RE...>, 0.12 and 0.13 were last year and 0.20 is not released), largely for performance reasons.

Please don’t just call the thin-std approach “problematic” without acknowledging that the alternative is at least as problematic, just with a different set of caveats.

Many of these have their own tracking issues on the path to v1.0. For example see this one for base64.

I see base64. If the standard library has base64 encoding, go ahead and use it. But as a third-party dependency? Again, base64 encoding and decoding is trivial. I've written this a few times myself. It's not worth a dependency.

Let's say crate B depends on crate A with a pinned dependency, and uses one of its types in a public interface.

Crate C depends on them both. It now can't bring in updates to A until B does, and when B updates that's a breaking change, so it better bump its major version.

Take a look at this teick, for example, for foundational crates updating their major version: https://github.com/dtolnay/semver-trick

Now imagine that being an issue every single patxh update.

Rust modules require a tiny bit more definition up-front, but they neatly decouple the module hierarchy from file layout so you can reorganize code however you like in future, and they support very fine grained control of privacy (such as being able to say pub(super) and pub(crate)). In extreme cases, you can even re-export symbols from one module in another without it counting as a breaking change, so you have even more options for evolving your project without breaking existing consumers. Look at the the semver trick as an example of how powerful this can be and how much freedom it gives library implementors. (And even if you're only a library consumer, wouldn't you rather be consuming libraries by implementors that had more freedom and power?)

A library author concerned about this can use the semver trick. TL;DR: if your current version is 0.42, you can do a 1.0 release, then do a 0.43 release that depends upon your 1.0 release and re-exports all the symbols.

I mean for all the parts of the standard library that do not change, one could presumably use the semver-trick.

The semver trick can help with libraries at least when they go to unify the ecosystem. Release new versions that replicate previous APIs in a compatible way while moving to the standard library implementation.

Because you still run into the problem that's been seen when various important crates upgraded and either didn't use the semver trick or had downstream crates specifying Cargo.toml version requirements too narrowly for it to be effective.

It's called the "semver-trick" [1].

[1]: https://github.com/dtolnay/semver-trick

The JVM is 114MiB on my machine. A near-minimal ggez program in debug mode is about 100MiB,¹ and ggez is small for a Rust application library. When you start getting into the 300s of dependencies (i.e. every time I've ever got beyond a trivial desktop application), you're lucky if your release build is less than 100MiB.

Sure, I could probably halve that by forking every dependency so they aren't duplicating versions, but that's a lot of work. (It's a shame Rust doesn't let you do conditional compilation based on dependency versions, or this would be a lot easier. As it is, we have to resort to the Semver trick: https://github.com/dtolnay/semver-trick/ — not that many people do that, so it's functionally useless.)

¹: I can get it down to around 8MiB with release mode, lto etc., but that significantly increases the build time and only about halves the weight of the intermediate build files.