Cargo VS min-sized-rust

cargo, rustfmt, clippy, rust-analyzer, and Rust’s robust unit testing capabilities together form a powerful ecosystem for managing large-scale projects like Spin.

That work is being tracked in https://github.com/rust-lang/cargo/issues/5931

Someone has taken up the work on this though there are some foundational steps first.

1. We need to delineate intermediate and final build artifacts so people have a clearer understanding in `target/` what has stability guarantees (implemented, awaiting stabilization).

2. We then need to re-organize the target directory from being organized by file type to being organized by crate instance.

3. We need to re-do the file locking for `target/` so when we share things, one cargo process won't lock out your entire system

4. We can then start exploring moving intermediate artifacts into a central location.

There are some caveats to this initial implementation

- To avoid cache poisoning, this will only items with immutable source that and an idempotent build, leaving out your local source and stuff that depends on build scripts and proc-macros. There is work to reduce the reliance on build scripts and proc-macros. We may also need a "trust me, this is idempotent" flag for some remaining cases.

- A new instance of a crate will be created in the cache if any dependency changes versions, reducing reuse. This becomes worse when foundation crates release frequently and when adding or updating a specific dependency, Cargo prefers to keep all existing versions, creating a very unpredictable dependency tree. Support for remote caches, especially if you can use your project's CI as a cache source, would help a lot with this.

> It seems wild to consider such intermediate files as part of public API. Someone relying on it does not automatically make it a breaking change if it’s not documented.

To find what is considered an intermediate vs a final artifact from cargo, you need to check out https://doc.rust-lang.org/cargo/reference/build-cache.html

We are working on making this clearer with https://github.com/rust-lang/cargo/issues/14125 where there will be `build.build-dir` (intermediate files) and `build.target-dir` (final artifacts).

When you do a `cargo build` inside of a library, like `clap`, you will get an rlip copied into `build.target-dir` (final artifacts). This is intended for integration with other build systems. There are holes with this workflow though but identifying all of the relevant cases for what might be a "safe" breakage is difficult.

See https://github.com/rust-lang/cargo/issues/13897 and https://github.com/rust-lang/cargo/issues/13897#issuecomment... .

Powerful tooling: Cargo simplifies dependency management, builds, and testing.

What I mean is that in open source, things only get done by people motivated to do them. Nobody has ever even asked for darcs support: https://github.com/rust-lang/cargo/issues?q=is%3Aissue%20sta...

So the lack of darcs isn’t because the Cargo folks think it’s bad or something. Just that things don’t get added just because.

Re quirks, sure, that’s why rustc and cargo are different. You don’t have to use Cargo. Meta does not, the Linux kernel does not.

One of the reasons why user PRs are so frequent is that Rust itself is a pretty straightforward language to contribute to. While the language itself is on the complex side (to say the least), its strict compiler and single package manager make it relatively manageable to run and test any changes and to be confident that what you've submitted will work as expected.

> That’s because, while cargo is great at building things, it is very simplistic at installing them. Cargo wants everything in a few neat binaries, and that isn’t our use case. Fish has about 1200 .fish scripts (961 completions, 217 associated functions), as well as about 130 pages of documentation (as html and man pages), and the web-config tool and the man page generator (both written in python).

Our issue for this is https://github.com/rust-lang/cargo/issues/2729

Personally, I lean away from Cargo expanding into these use cases and prefer another tool being implemented on top. I've written more about this at https://epage.github.io/blog/2023/08/are-we-gui-build-yet/

Is an explicit, safe and less error prone way of doing it.

...and one that doesn't walk us down the road of (see the linked thread) the obvious desire people are going to have sooner or later to cache binary builds instead of building locally, and turn `cargo install` into some kind of binary application distribution application or app store.

If you don't believe me, read that thread, and the linked thread.

[1] - https://github.com/rust-lang/cargo/issues/13994#issuecomment...

This is a good guide on building small Rust binaries: https://github.com/johnthagen/min-sized-rust

This talks about going to extreme lengths on making the smallest Rust binary possible, 400 bytes when it was written, https://darkcoding.net/software/a-very-small-rust-binary-ind...

The thing is, you lose a lot of nice features when you do this, like panic unwinding, debug symbols, stdlib… for kernel and some embedded development it’s definitely important, but for most use cases, does it matter?

Oh it was 137, haha. I will link you to this older comment of mine: https://news.ycombinator.com/item?id=29408906

See also https://github.com/johnthagen/min-sized-rust

Are you sure? If so then this is awesome news, but I'm a bit confused; the commit in that min-sized-rust repo adding `build-std` to the README was merged in August 2021: https://github.com/johnthagen/min-sized-rust/pull/30

Are you saying that at that point the feature still hadn't "landed in Rust nightly" until recently? If so then what's the difference between a feature just being available in Rust nightly, vs having "landed"?

Rust binaries are by default nowhere close to 500MB. If they are not small enough for you, you can try https://github.com/johnthagen/min-sized-rust. By avoiding the formatting machinery and using `panic_immediate_abort` you can get about the size of C binaries.

A useful reference: https://github.com/johnthagen/min-sized-rust

That must be without stripping. Also there are ways to reduce binary size. See e.g. [min-sized-rust](https://github.com/johnthagen/min-sized-rust). I've gotten stripped binaries of small cli utils less than 400KiB without doing anything special, less than 150 KiB by customizing profile settings and compressing with upx, and less than 30 KiB by replacing the std with the libc as the link shows. Haven't tried with fltk though...

This is not quite what you're asking, but it does also address the underlying concern: https://github.com/johnthagen/min-sized-rust