Rust Dependencies Scare Me

1. rfcs

   1 690 6,214 9.1 Markdown

   RFCs for changes to Rust

   

   Actually, a proposal for exactly this was published yesterday: https://github.com/rust-lang/rfcs/pull/3810

   It's unfortunate that the response so far hasn't been very positive

2. Stream

   getstream.io featured

   Stream - Scalable APIs for Chat, Feeds, Moderation, & Video. Stream helps developers build engaging apps that scale to millions with performant and flexible Chat, Feeds, Moderation, and Video APIs and SDKs powered by a global edge network and enterprise-grade infrastructure.

   

3. dotenv

   2 2 594 0.0 Rust

   Library to help supply environment variables in testing and development (by dotenv-rs)

   

   The maintainers themselves give this warning in the repo's README, so even if it were maintained, it still wouldn't be production ready.

   > Achtung! This is a v0.* version! Expect bugs and issues all around. Submitting pull requests and issues is highly encouraged!

   https://github.com/dotenv-rs/dotenv

4. swc

   3 160 32,467 9.9 Rust

   Rust-based platform for the Web

   

   I once wanted to contribute to the popular swc project (https://github.com/swc-project/swc). I cloned the repo, ran build, and a whooping 20GB was gone from my disk. The parser itself (https://github.com/swc-project/swc/blob/main/crates/swc_ecma...) has over a dozen dependencies, including serde.

   Meanwhile, the heaviest JavaScript parser implemented in JavaScript is more lightweight.

   I decided that I should leave this project alone and spend my time elsewhere.

5. Moby

   4 239 70,070 10.0 Go

   The Moby Project - a collaborative project for the container ecosystem to assemble container-based systems

   

   Yeah, while I’ve seen some great libraries that follow the practice of minimizing their dependencies, I’m a bit annoyed with the amount of dependencies that docker will bring along [1]. I’ve been on the lookout for alternatives for my docker needs, but the state of podman, buildah and some others that I checked is similar. They all bring in roughly the same number of dependencies… if anyone knows of a stripped down Go lib that can be used to build from a Dockerfile, pull, and run a container, I would be grateful for any suggestions. Heck docker / moby isn’t even using go.mod proper.

   [1] https://github.com/moby/moby/blob/master/vendor.mod

6. chi

   5 110 20,081 6.9 Go

   lightweight, idiomatic and composable router for building Go HTTP services

   

7. httprouter

   6 41 16,916 4.7 Go

   A high performance HTTP request router that scales well

   

8. mux

   7 92 17,948 2.6 Go

   Discontinued A powerful HTTP router and URL matcher for building Go web servers with 🦍

   

9. InfluxDB

   www.influxdata.com featured

   InfluxDB – Built for High-Performance Time Series Workloads. InfluxDB 3 OSS is now GA. Transform, enrich, and act on time series data directly in the database. Automate critical tasks and eliminate the need to move data externally. Download now.

   

10. pflag

    8 15 2,593 6.8 Go

    Drop-in replacement for Go's flag package, implementing POSIX/GNU-style --flags.

    

11. uuid

    9 21 5,722 5.1 Go

    Go package for UUIDs based on RFC 4122 and DCE 1.1: Authentication and Security Services. (by google)

    

12. Telegraf

    10 114 15,741 9.9 Go

    Agent for collecting, processing, aggregating, and writing metrics, logs, and other arbitrary data.

    

    Wow, that's massive. I guess it's inevitable that a popular piece of open-source software for end-users will be compelled to accrue dependencies due to popular demand for features that require them.

    I feel Telegraf made a good compromise: out of the box, it comes with a _ton_ of stuff[1] to monitor everything, but they make it possible to build only with pieces that you need via build tags, and even provide a tool to extract said tags from your telegraf config[2]. But lots of supply-chain security stuff assume everything in go.mod is used, so that can results in a lot of noise.

    [1] https://github.com/influxdata/telegraf/blob/master/go.mod

13. wuffs

    11 95 4,547 8.1 C

    Wrangling Untrusted File Formats Safely

    

    You want a special purpose language.

    In your particular example of image loading, you want WUFFS. https://github.com/google/wuffs

    In WUFFS most programs are impossible. Their "Hello, world" doesn't print hello world because it literally can't do that. It doesn't even have a string type, and it has no idea how to do I/O so that's both elements of the task ruled out. It can however, Wrangle Untrusted File Formats Safely which is its sole purpose.

    I believe there should be more special purpose languages like this, as opposed to the General Purpose languages most of us learn. If your work needs six, sixteen or sixty WUFFS libraries to load different image formats, that's all fine because categorically they don't do anything outside their box. Yet, they're extremely fast because since they can't do anything bad by definition they don't need those routine "Better not do anything bad" checks you'd write in a language like C or the compiler would add in a language like Rust, and because they vectorize very nicely.

14. cargo-vendor-filterer

    12 3 44 5.3 Rust

    Tool to `cargo vendor` with filtering

    

    > I can't rewrite the world, an async runtime and web server are just too difficult and take to long for me to justify writing for a project like this (although I should eventually just for a better understanding).

    I did this and it only solved half of the bloat:

    https://crates.io/crates/safina - Safe async runtime, 6k lines

    https://crates.io/crates/servlin - Modular HTTP server library, threaded handlers and async performance, 8k lines.

    I use safina+servlin and 1,000 lines of Rust to run https://www.applin.dev, on a cheap VM. It serves some static files, a simple form, receives Stripe webooks, and talks to Postgres and Postmark. It depends on some heavy crate trees: async-fs, async-net, chrono, diesel, rand (libc), serde_json, ureq, and url.

    2,088,283 lines of Rust are downloaded by `cargo vendor` run in the project dir.

    986,513 lines using https://github.com/coreos/cargo-vendor-filterer to try to download only Linux deps with `cargo vendor-filterer --platform=x86_64-unknown-linux-gnu`. This still downloads the `winapi` crate and other Windows crates, but they contain only 22k lines.

    976,338 lines omitting development dependencies with `cargo vendor-filterer --platform=x86_64-unknown-linux-gnu --keep-dep-kinds=normal`.

    754,368 lines excluding tests with `cargo vendor-filterer --platform=aarch64-apple-darwin --exclude-crate-path='*#tests' deps.filtered`.

    750k lines is a lot to support a 1k-line project. I guess I could remove the heavy deps with another 200 hours of work, and might end up with some lean crates. I've been waiting for someone to write a good threaded Rust Postgres client.

15. applin-rails-demo

    13 5 2 7.7 Ruby

    Example of how to use applin-rails.

    

    > I can't rewrite the world, an async runtime and web server are just too difficult and take to long for me to justify writing for a project like this (although I should eventually just for a better understanding).

    I did this and it only solved half of the bloat:

    https://crates.io/crates/safina - Safe async runtime, 6k lines

    https://crates.io/crates/servlin - Modular HTTP server library, threaded handlers and async performance, 8k lines.

    I use safina+servlin and 1,000 lines of Rust to run https://www.applin.dev, on a cheap VM. It serves some static files, a simple form, receives Stripe webooks, and talks to Postgres and Postmark. It depends on some heavy crate trees: async-fs, async-net, chrono, diesel, rand (libc), serde_json, ureq, and url.

    2,088,283 lines of Rust are downloaded by `cargo vendor` run in the project dir.

    986,513 lines using https://github.com/coreos/cargo-vendor-filterer to try to download only Linux deps with `cargo vendor-filterer --platform=x86_64-unknown-linux-gnu`. This still downloads the `winapi` crate and other Windows crates, but they contain only 22k lines.

    976,338 lines omitting development dependencies with `cargo vendor-filterer --platform=x86_64-unknown-linux-gnu --keep-dep-kinds=normal`.

    754,368 lines excluding tests with `cargo vendor-filterer --platform=aarch64-apple-darwin --exclude-crate-path='*#tests' deps.filtered`.

    750k lines is a lot to support a 1k-line project. I guess I could remove the heavy deps with another 200 hours of work, and might end up with some lean crates. I've been waiting for someone to write a good threaded Rust Postgres client.

16. cargo-geiger

    14 33 1,496 4.6 Rust

    Detects usage of unsafe Rust in a Rust crate and its dependencies.

    

    I love this idea and I hope I get to work on it someday. I think it could be possible in Rust with a linter, something like https://github.com/geiger-rs/cargo-geiger . The Rust compiler has some unsoundness issues such as https://github.com/rust-lang/rust/issues/84366 . Those would need fixing or linter coverage.

17. rust

    15 2,856 104,879 10.0 Rust

    Empowering everyone to build reliable and efficient software.

    

    I love this idea and I hope I get to work on it someday. I think it could be possible in Rust with a linter, something like https://github.com/geiger-rs/cargo-geiger . The Rust compiler has some unsoundness issues such as https://github.com/rust-lang/rust/issues/84366 . Those would need fixing or linter coverage.

18. miqt

    16 5 474 9.8 C++

    MIT-licensed Qt bindings for Go

    

    > As far as I'm aware, LTO completely solves this from a binary size perspective.

    I wouldn't say completely. People still sometimes struggle to get this to work well. For example:

    https://github.com/mappu/miqt/issues/147

19. capslock

    17 4 935 8.4 Go

    

    Capslock sort of does this with go https://github.com/google/capslock

20. CPM.cmake

    18 42 3,541 5.9 CMake

    📦 CMake's missing package manager. A small CMake script for setup-free, cross-platform, reproducible dependency management.

    

    I also don't understand the CMake hate. Modern CMake (3.14+) is just around 10 lines to build basic sources/libraries/executables. And you can either use CMake FetchContent or use CPM https://github.com/cpm-cmake/CPM.cmake to fetch dependencies. No third-party tool like vcpkg or conan is needed.

21. bluefin

    19 5 111 9.2 Haskell

    

    Yes, there is a sense in which Haskell's "effect systems" are "capability systems". My effect system, Bluefin, models capabilities as values that you explicitly pass around. You can't do I/O unless you have the "IOE" capability, for example.

    https://hackage.haskell.org/package/bluefin

22. rebar

    20 29 255 6.6 Python

    A biased barometer for gauging the relative speed of some regex engines on a curated set of tasks.

    

    To be fair I think Rust has very healthy selection of options for both, with Serde and Reqwest/Hyper being de-facto standard.

    Rust has other challenges it needs to overcome but this isn't one.

    I'd put go way behind both C#/F# and Rust in this area. It has lacking tooling in odd areas it's expected to be strong at like gRPC and the serialization story in Go is quite a bit more painful and bare bones compared to what you get out of System.Text.Json and Serde.

    The difference is especially stark with Regex where Go ships with abysmally slow engine (because Go the language does not allow writing sufficiently fast code in this area) where-as both Rust and C# have top of the line implementations in each which beat every other engine save for Intel Hyperscan[0].

    [0]: https://github.com/BurntSushi/rebar?tab=readme-ov-file#summa... (note this is without .NET 9 or 10 preview updates)

23. go

    21 2,276 128,800 10.0 Go

    The Go programming language

    

    > (because it does not allow writing sufficiently fast code in this area at this moment)

    I don't think that's why. Or at least, I don't think it's straight-forward to draw that conclusion yet. I don't see any reason why the lazy DFA in RE2 or the Rust regex crate couldn't be ported to Go[1] and dramatically speed things up. Indeed, it has been done[2], but it was never pushed over the finish line. My guess is it would make Go's regexp engine a fair bit more competitive in some cases. And aside from that, there's tons of literal optimizations that could still be done that don't really have much to do with Go the language.

    Could a Go-written regexp engine be faster or nearly as fast because of the language? Probably not. But I think the "implementation quality" is a far bigger determinant in explaining the current gap.

    [1]: https://github.com/golang/go/issues/11646

    [2]: https://github.com/matloob/regexp

24. regexp

    22 1 38 - Go

    go regexp with RE2 DFA matcher port - golang.org/cl/12081 (by matloob)

    

    > (because it does not allow writing sufficiently fast code in this area at this moment)

    I don't think that's why. Or at least, I don't think it's straight-forward to draw that conclusion yet. I don't see any reason why the lazy DFA in RE2 or the Rust regex crate couldn't be ported to Go[1] and dramatically speed things up. Indeed, it has been done[2], but it was never pushed over the finish line. My guess is it would make Go's regexp engine a fair bit more competitive in some cases. And aside from that, there's tons of literal optimizations that could still be done that don't really have much to do with Go the language.

    Could a Go-written regexp engine be faster or nearly as fast because of the language? Probably not. But I think the "implementation quality" is a far bigger determinant in explaining the current gap.

    [1]: https://github.com/golang/go/issues/11646

    [2]: https://github.com/matloob/regexp

25. prost

    23 15 4,320 8.7 Rust

    PROST! a Protocol Buffers implementation for the Rust Language

    

    People are paid to work on standard libraries and there’s a whole process behind developing and releasing this software.

    Tokio on the other hand is the library whose maintainer decided to download a binary blob during build: https://github.com/tokio-rs/prost/issues/562 https://github.com/tokio-rs/prost/issues/575

    Good luck catching such issues across dozens of crates.

26. SaaSHub

    www.saashub.com featured

    SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives

    

Suggest a related project