serde VS too-many-lists

Serde Documentation: Comprehensive guide and reference for using Serde, Rust's framework for serializing and deserializing data.

Not just for hardware support: https://github.com/serde-rs/serde/issues/2538

tokio / hyper / toml / serde / serde_json / json5 / console

serde - Serializing and Deserializing Rust data structures

> If there are any Rust experts around...what am I missing?

Probably more good examples and possibly more ecosystem.

A first big decision is if you're going no-std or not, and if you end up in the no-std world the ecosystem shrinks substantially. If you're building a kernel that's probably not that much of a problem - the same shrinkage occurs for C/C++ - many such projects bootstrap with nearly zero ecosystem anyway.

The examples side is a bigger problem - I've recently been able to watch some of my more curmudgeonly C friends give it a good dive, and after an initial hump they're fairly happy with the core language. They still have regular issues with the ecosystem when they run into, in their words "web dev crap", which comes up even in the stdlib sometimes - a bugbear a while ago coming up down some error handling code paths. They attempted to send patches and hit nebulous arguments against the correctness target, which were largely born of misunderstandings of posix. This kind of thing can come up anywhere, if you take a dependency on some fancy IO abstraction that happens to be written in C, and you take it somewhere "novel" like a BSD, you might well run into the same. The point here is that _examples_ and _exercise_ of these tasks are the things that are going to shake more of this kind of thing out. At the same time though, it's important to reiterate that if you're on the nostd path, then largely you're on your own, which is equivalent to just gcc bare, and this kind of thing generally doesn't come up.

> Or is this a serious proposal about the future of operating systems and other low level infrastructure code?

This is a serious proposal. The outcome is really strong along key axes of correctness and safety. Those of us who've done it (e.g. Fuchsia, where I was) have been able to observe these benefits relative to history with the same teams using other languages (C, C++). We're professional engineers, these statements aren't coming from a place of craziness. The Android team have been writing about their journey: https://security.googleblog.com/search/label/rust

> Do you just program everything in unsafe mode?

Absolutely not. A good amount of bootstrapping effort has been going in across the ecosystem to make it ever easier to avoid unsafe. To take one slice of examples, there's crates that are designed to help you avoid copies while also avoiding dropping to unsafe - they provide tools for automatic structural analysis of the mapping boundary to make it easier to assert the relevant guarantees. Examples: https://docs.rs/zerocopy/latest/zerocopy/ (came out of Fuchsia), https://github.com/serde-rs/serde/releases/tag/v1.0.0 (serde is commonly used, but has more constraints here), https://rkyv.org/rkyv.html (not sure of prominence, but I hear people talk about it).

These kinds of tools get you a long way toward substantially safer code, without needing to think or audit nearly as much. We know that's important, we have plenty of data that demonstrates how important it is, and lately now, we have data that shows how effective it is too (see the aforementioned Android posts).

> What about runtimes?

They're out there, it depends what level of abstraction you're looking for, runtimes means different things to different people. For embedded there's typically a lot more focus on providing libraries rather than a whole runtime framework, so there are crates for a number of soc types out there which are well used, like https://docs.rs/cortex-m/latest/cortex_m/, or it's sibling minimal runtime crate https://docs.rs/cortex-m-rt/latest/cortex_m_rt/. As you get higher level, if you want to do more of the systems level interaction yourself there are a good number of options to choose from along the lines of reactor systems to get you to functional async executors that will build with nostd.

> It seems to me that Rust isn't even really intended to compete with C for the use cases in which C is dominant in 2023. Every indication is that for "serious Rust in production programming" it's mostly a C++ crowd.

The challenge here is that this is an inverted view - it's a C programmers view of C++ being ported over to Rust, and it distorts how the world looks. That doesn't actually apply to the other sides _intent_. Yes, Rust provides a lot more abstraction capabilities than C does, and in that specific regard it has some coarse similarity to C++. It's definitely _possible_ for someone to way off the deep end and produce obscure abstractions around things that a more reductionist bias is going to hate - and you can totally ignore those things and have a great time with the language. There are some really nice things in there which anyone will enjoy if they come at it with an open mind, things like enums and pattern matching, the rich and efficient iterator library, the crates tooling, configuration macros, and so on. There's a lot to love, and they're not things that C++ did well, and comparing it to C++ discards those considerations spuriously.

> Zig has sort of filled that similar space and seems to take the concerns of C programmers more seriously and the team has an attitude more in line with the C culture than the Rust team does

Zig is interesting in it's own right, and on a very surface level it is more similar to C, though this surface level is really "zigs stdlib has terrible and inconsistent short names similar to libc and posix", which isn't really a good measure of anything in particular. It's perfectly functional along this axis and par for the course at the systems programming level.

The toolchain approach has a lot more of a "hacking for hackers" feel, so when you hit bugs and so on it's very typical for folks to be patching their stdlib locally for a while or building their own toolchain to overcome the problems. I spent a little bit of time there recently with building ghostty on windows and was regularly messing with my toolchain and stdlib to make forward progress. Along these lines it's also much less complete - which is largely a function of being much newer, but you can take a ton of rust projects today, particularly things in the GUI space and build them for every major target platform with nearly zero effort. Zig is very very far off that, and there's going to be a need for a lot better platform level abstractions to get there. Rust did a great job with platform abstractions, which sadly was best documented in an anti-go inflammatory articles, but the point stands and if more generalized stands against zig at times too (https://fasterthanli.me/articles/lies-we-tell-ourselves-to-k... and other similar rants he wrote), though not all points port over.

The LLVM removal kind of move is somewhat enabled by this looser approach, which is also helped by the kinds of users the language has, and the smaller ecosystem. Another way of putting that might be "this is the right time to do this", as doing it later might lead to far more user pain and community noise or negativity. It's great for the world as we need more diversity, but it's also not all roses. At my current work we tried out Zig for hermetic cross-builds something that a lot of people tout as a strength. What we found was that the intrinsics that were written in pure Zig were sufficiently far behind libgcc/compiler-rt that it did not have sufficient performance for our use case - literally the binary couldn't handle our production load. Again, this is the kind of thing that can and likely will improve with time, hell if it was a priority I would have done it, but we had other solutions. Point is it's not as simple as a "this vs that" outcome, these moves have long running implications that may or may not affect a particular target - as an example it didn't really harm ghostty at all.

When you talk about culture each of these ecosystems has it's own dominant culture and a wide set of subcultures. How you choose to integrate with those, if you do at all, is up to you. Some might be more attractive for sure, and some might provide a different risk profile for different use cases as well.

Just off the cuff if I was scaling up a team for a professional project with a long lifecycle, I'd probably lean toward rust right now as it has a good balance of stable evolution and production readiness, without being anywhere near as stagnant as C++ despite much effort to move the needle. If I was in a really hacker mood where I just want to twiddle and mess with stuff, I'm not excessively performance sensitive (beyond the general order of magnitude that native compilation and near zero abstractions gets you), and my team is going to remain small and expert "everyone cracks open the source" folks, then I might pick Zig. These days I don't have many good reasons to pick C anymore. If it's patching a pre-existing thing there's no choice of course, but other than that it's mostly going to be "I'm throwing a 30 minute build onto arduino and don't wanna go off the beaten path for this project" kind of thing.

serde: Serialization and deserialization framework.

Note that the pre-compiled binary blob the blog post is referring to has since been removed [0].

[0]: https://github.com/serde-rs/serde/pull/2590

You seem to have a preset opinion, and I'm not sure you are interested in re-evaluating it. So this is not written to change your mind.

I've developed production code in C, C++, Rust, and several other languages. And while like pretty much everything, there are situations where it's not a good fit, I find that the solutions tend to be the most robust and require the least post release debugging in Rust. That's my personal experience. It's not hard data. And yes occasionally it's annoying to please the compiler, and if there were no trait constraints or borrow rules, those instances would be easier. But way more often in my experience the compiler complained because my initial solution had problems I didn't realize before. So for me, these situations have been about going from building it the way I wanted to -> compiler tells me I didn't consider an edge case -> changing the implementation and or design to account for that edge case. Also using one example, where is Rust is notoriously hard and or un-ergonomic to use, and dismissing the entire language seems premature to me. For those that insist on learning Rust by implementing a linked list there is https://rust-unofficial.github.io/too-many-lists/.

Advent of Code was okay until I encounterd a problem that required a graph, tree or linked list to solve, where I hit a wall. Most coding exercises are similar--those requiring arrays and hashmaps and sets are okay, but complex data structures are a PITA. (There is an online course dedicated to linked lists in Rust but I couldn't grok it either). IMO you should simply skip problems that you can't solve with your current knowledge level and move on.

I feel obligated to point to the original cannon literature: https://rust-unofficial.github.io/too-many-lists/

I started learning Rust and like how the compiler is fussy about things. My plan was to implement the data structures I knew, but I got stuck at the singly linked list's remove() method. I've read the book as well, but I have no clue how to simplify this further:

My impression from the article is that Zig provides several different hashtables and not all of them are broken in this way.

This reminds me of Aria's comment in her Rust tutorial https://rust-unofficial.github.io/too-many-lists/ about failing to kill LinkedList. One philosophy (and the one Rust chose) for a stdlib is that this is only where things should live when they're so commonly needed that essentially everybody needs them either directly or to talk about. So, HashTable is needed by so much otherwise unrelated software that qualifies, BloomFilter, while it's real useful for some people, not so much. Aria cleaned out Rust's set of standard library containers before Rust 1.0, trying to keep only those most people would need. LinkedList isn't a good general purpose data structure, but, it was too popular and Aria was not able to remove it.

Having multiple hash tables feels like a win (they're optimized for different purposes) but may cost too much in terms of the necessary testing to ensure they all hit the quality you want.

> Cyclic references can be dealt with runtime safety checks too - like Rc and Weak.

Indeed. Starting out with code sprinkled with Rc, Weak, RefCell, etc is perfectly fine and performance will probably not be worse than in any other safe languages. And if you do this, Rust is pretty close to those languages in ease of use for what are otherwise complex topics in Rust.

A good reference for different approaches is Learn Rust With Entirely Too Many Linked Lists https://rust-unofficial.github.io/too-many-lists/

Second, once you've finished something introductory like The Book, read Learning Rust With Entirely Too Many Linked Lists. It really helped me to understand what ownership and borrowing actually mean in practical terms. If you don't mind paying for learning materials, a lot of people recommend Programming Rust, Second Edition by Blandy, Orendorff, and Tindall as either a complement, follow-up, or alternative to The Book.

Learning Rust With Entirely Too Many Linked Lists which highlights a lot of the differences with how you need to structure your code in Rust compared to other languages.