mold VS cargo-chef

I think this is unlikely to gain traction. I say that no to discourage you, just to explain.

- The community has an instinctive distrust of closed source or a compiler from an untrusted source. If you’re familiar with the Trusting Trust attack you’ll understand why.

- Dev tools in every language ecosystem are almost always free, unless they involve some kind of hosting. People aren’t used to opening their wallets. Look the experience of the guy who built the mold linker(https://github.com/rui314/mold). Far superior to the state of art, improves incremental compiles a lot, widely applicable across ecosystems (C, C++, Rust), CPU architectures and Operating Systems. You don’t even have to modify your compiler, just need to point to his linker. He’s even giving it away for free for personal use. But still, almost no one uses it. The inertia of the established options is really high.

- It’s not complex enough. Think about the complexity involved in the cranelift backend. No one can seriously recreate the efforts of bjorn3. If we could have, we would have. But the idea idea here can be recreated, especially by the experts who already built incremental compilation into rustc.

- But if your solution is truly complex, like the parallel frontend, the burden of maintaining a fork would be too high. You’d have to spend all your time rebasing.

Again I’m not trying to discourage you, just stating the difficulties of making a business in the dev tools space. You would be better off contributing this excellent work to the community and trying a different tack.

I initially thought this would be about the mold linker (https://github.com/rui314/mold)

I assume this submission is trying to highlight the specific message (2023-01-24) : https://github.com/rui314/mold/issues/190#issuecomment-14028...

Fyi... the author wrote a more expansive blog post about selling dev tools a few months later (2023-06-06) and there was a related HN thread about it: https://news.ycombinator.com/item?id=36225016

Loongson's LoongArch CPU has been supported. (03b1a1c)

I'm amazed at how quickly the author responds to requests: https://github.com/rui314/mold/issues/1057

From the report to the fix in less than two days.

I'm not sure how competitive it will be with lld, especially if we consider ThinLTO (which takes multiple minutes on 64-core machine) - it can make the advantages of mold insignificant.

What other solutions have you tried for the problem of slow linking? You haven't even said which linker and what flags you're using. I haven't actually tried it, but the author of gold has an even faster linker called mold: https://github.com/rui314/mold

> Mold did it first, though: https://github.com/rui314/mold

Before LLD?

CI time. Do you want to micromanage your own docker images for all your CI? Great! If not, yes you do. In fact, you want to manage a docker image to build a docker image to use for CI. Use cargo-chef to prepare a build image with your dependencies pre-built if you want to do fine-grained build/test pipelines. Oh also, there's no jUnit test report generation, that was killed off today. (YES, SORRY, I'm still salty.)

Have you looked at cargo-chef? It supposedly speeds up compilation times if you're using Docker.

A tool already exists for this called Cargo-chef, and it works extremely well.

At work I've use Dockerfile and cargo-chef to improve build times. You can also look into buildkit cache mounts, but this approach is rarely super effective on hosted CI because they start from scratch on most runs. In the context of Rust specifically you may also see the target directory reflect unbounded data growth if it's reused over and over across revisions. because cargo by default won't expire older intermediate artifacts. Cargo-sweep can help with that but I wouldn't pursue this in a CI effort. This will affect both "native" builds and buildkit cache mounts if you're persisting the target directory.

We create an empty Rust binary with cargo new, this is a simple way to get Docker layer caching to work. For a more robust solution, you may want to check out cargo-chef.

If this ends up being a cleaner/easier way to having to workaround super expensive rebuilds for Rust given cache + deps compared to this https://github.com/LukeMathWalker/cargo-chef , reading this thread will have been a huge win for me (and hopefully others).

Whether introducing Bazel is easier/worth it, subjective I guess.

I did a talk (slides here) about this a few years ago, it took a bit of work to get the build caching working with cargo. As others have pointed out, there is now cargo chef to solve this problem so you probably don't have to deal with the issues I saw, but I thought it still might be helpful context.

I'm curious if you've tried out cargo-chef, I've had some decent improvements with it but I wonder how it stacks up to the sccache approach (don't have the time to try it out myself right now).

for CI have you tried to use buildkit persistent runners with caching + https://github.com/LukeMathWalker/cargo-chef ?

If you're deploying Rust with Docker I can tell you that cargo-chef is invaluable. With zero work it caches the dependency fetch and compilation steps. Most of the time the ens Docker deploy is closer to an incremental compile than full.