crate2nix VS cargo-chef

I'll use nixpkgs' buildRustPackage. There's a few other tools, my favorite being crate2nix, but we'll leave that to a future tutorial.

If you want to iterate with nix instead of cargo, crate2nix and cargo2nix provides more caching and more fine control over your dependencies. I haven't used these two so you would have to decide for yourself. You may also want to try out nocargo for something more experimental.

I'm yet to try it out, but from the blog post, the README and the source it appears that Crane builds all dependencies in one derivation (separately from the main crate). This means that if a dependency gets added, removed or changed, all dependencies of a crate will be rebuilt. This is in contrast with https://github.com/kolloch/crate2nix, which does build every dependency in a separate package, thus you don't need to rebuild other dependencies if you only change a small part of the tree.

Yes, you will have to package it if it's not already in nixpkgs.

The good news is once you learn how, it's basically trivial with crate2nix[0], which can autogenerate nix derivations from rust crates

[0] https://github.com/kolloch/crate2nix

From my quick reading of cargo2nix's webpage yes. https://github.com/kolloch/crate2nix has a workspaces section.

As for your two applications, they're both written in Rust, and I like https://github.com/kolloch/crate2nix as a way to package Rust crates with Nix. BTW, wezterm is already in nixpkgs!

I like the way crate2nix works. I have made a flake template for it here. Sometimes it requires a couple of overrides to fix some misbehaving crates (see https://github.com/balsoft/simple-osd-daemons/blob/master/flake.nix#L29 for an example of such overrides), but otherwise it's fantastic. It doesn't require any hash nonsense, it downloads and builds all the crates separately (unlike naersk or other solutions) so you get all the benefits of Nix (reproducibility and proper caching).

Crate2nix might help: https://github.com/kolloch/crate2nix

CI time. Do you want to micromanage your own docker images for all your CI? Great! If not, yes you do. In fact, you want to manage a docker image to build a docker image to use for CI. Use cargo-chef to prepare a build image with your dependencies pre-built if you want to do fine-grained build/test pipelines. Oh also, there's no jUnit test report generation, that was killed off today. (YES, SORRY, I'm still salty.)

Have you looked at cargo-chef? It supposedly speeds up compilation times if you're using Docker.

A tool already exists for this called Cargo-chef, and it works extremely well.

At work I've use Dockerfile and cargo-chef to improve build times. You can also look into buildkit cache mounts, but this approach is rarely super effective on hosted CI because they start from scratch on most runs. In the context of Rust specifically you may also see the target directory reflect unbounded data growth if it's reused over and over across revisions. because cargo by default won't expire older intermediate artifacts. Cargo-sweep can help with that but I wouldn't pursue this in a CI effort. This will affect both "native" builds and buildkit cache mounts if you're persisting the target directory.

We create an empty Rust binary with cargo new, this is a simple way to get Docker layer caching to work. For a more robust solution, you may want to check out cargo-chef.

If this ends up being a cleaner/easier way to having to workaround super expensive rebuilds for Rust given cache + deps compared to this https://github.com/LukeMathWalker/cargo-chef , reading this thread will have been a huge win for me (and hopefully others).

Whether introducing Bazel is easier/worth it, subjective I guess.

I did a talk (slides here) about this a few years ago, it took a bit of work to get the build caching working with cargo. As others have pointed out, there is now cargo chef to solve this problem so you probably don't have to deal with the issues I saw, but I thought it still might be helpful context.

I'm curious if you've tried out cargo-chef, I've had some decent improvements with it but I wonder how it stacks up to the sccache approach (don't have the time to try it out myself right now).

for CI have you tried to use buildkit persistent runners with caching + https://github.com/LukeMathWalker/cargo-chef ?

If you're deploying Rust with Docker I can tell you that cargo-chef is invaluable. With zero work it caches the dependency fetch and compilation steps. Most of the time the ens Docker deploy is closer to an incremental compile than full.