loom
miri
loom | miri | |
---|---|---|
14 | 121 | |
1,891 | 3,955 | |
3.1% | 2.3% | |
6.8 | 10.0 | |
7 days ago | 7 days ago | |
Rust | Rust | |
MIT License | Apache License 2.0 |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
loom
- Turmoil, a framework for developing and testing distributed systems
-
An Introduction to Lockless Algorithms
> Mutexes are very cheap in the uncontended case
It was a while ago I was deep into this mess so forgive any ignorance–but–iirc the thread-mutex dogma[1] has many pitfalls despite being so widely used. Primarily they’re easy to misuse (deadlocks, holding a lock across a suspend point), and have unpredictable performance because they span so far into compiler, OS and CPU territory (instruction reordering, cache line invalidation, mode switches etc). Also on Arm it’s unclear if mutices are as cheap because of the relaxed memory order(?). Finally code with mutices are hard to test exhaustively, and are prone to heisenbugs.
Now, many if not most of the above apply to anything with atomics, so lock-free/wait-free won’t help either. There’s a reason why a lot of concurrency is ~phd level on the theoretical side, as well as deeply coupled with the gritty realities of hardware/compilers/os on the engineering side.
That said, I still think there’s room for a slightly expanded concurrency toolbox for mortals. For instance, a well implemented concurrent queue can be a significant improvement for many workflows, perhaps even with native OS support (io_uring style)?. Another exciting example is concurrency permutation test frameworks[2] for atomics that reorder operations in order to synthetically trigger rare logical race conditions. I’ve also personally had great experience with the Golang race detector. I hope we see some convergence on some of this stuff within a few years. Concurrency is still incredibly hard to get right.
[1]: I say this only because CS degrees has preached mutices to as the silver bullet for decades.
[2]: https://github.com/tokio-rs/loom
-
Should atomics be unsafe?
Of course atomics are absolutely essential for some of the libraries we take for granted, such as Arc and Tokio. But if you start reading the code and comments and issues and PRs around code like that, you'll see how much work it took to mature them to the point we can now rely on them. That's why tools like Loom exist.
-
Best tool to find deadlocks (in async code)
loom and shuttle can help you narrow down the problem.
-
Does Rust not need extra linting and sanitizing tools like C++?
Unless you are writing unsafe code, you generally don't need to use sanitizers. If you do write unsafe code, checking it with a sanitizer would be a great idea. Two most useful tools here I think are miri and loom.
-
The Deadlock Empire
https://github.com/tokio-rs/loom perhaps? It also models weak memory reordering, but takes some work to integrate into existing apps.
For triggering race conditions in compiled binaries, you could try https://robert.ocallahan.org/2016/02/introducing-rr-chaos-mo....
-
What could Go wrong with a mutex? (A Go profiling story)
There is Loom[1] (part of the Tokio project) for exhaustively testing multithreaded code. Though as far as I can tell it is designed for debugging threads, not async tasks.
[1] https://github.com/tokio-rs/loom
-
Cooptex - Deadlock-free Mutexes
That tool seems similar to https://github.com/tokio-rs/loom, insofar as detecting potential locking errors. These are useful during development, but could still miss production cases (as dev never perfectly matches production). This crate is meant to not have to worry about possibly deadlocking.
-
A bug that doesn’t exist on x86: Exploiting an ARM-only race condition
Rust doesn't catch memory ordering errors, which can result in behavioral bugs in safe Rust and data races and memory unsafety in unsafe Rust. But Loom is an excellent tool for catching ordering errors, though its UnsafeCell API differs from std's (and worse yet, some people report Loom returns false positives/negatives in some cases: https://github.com/tokio-rs/loom/issues/180, possibly https://github.com/tokio-rs/loom/issues/166).
- Multicore OCaml: April 2021
miri
-
Bytecode VMs in Surprising Places
Miri [0] is an interpreter for the mid-level intermediate representation (MIR) generated by the Rust compiler. MIR is input for more processing steps of the compiler. However miri also runs MIR directly. This means miri is a VM. Of course it's not a bytecode VM, because MIR is not a bytecode AFAIK. I still think that miri is a interesting example.
And why does miri exist?
It is a lot slower. However it can check for some undefined behavior.
[0]: https://github.com/rust-lang/miri
-
RFC: Rust Has Provenance
Provenance is a dynamic property of pointer values. The actual underlying rules that a program must follow, even when using raw pointers and `unsafe`, are written in terms of provenance. Miri (https://github.com/rust-lang/miri) represents provenance as an actual value stored alongside each pointer's address, so it can check for violations of these rules.
Lifetimes are a static approximation of provenance. They are erased after being validated by the borrow checker, and do not exist in Miri or have any impact on what transformations the optimizer may perform. In other words, the provenance rules allow a superset of what the borrow checker allows.
- Mir: Strongly typed IR to implement fast and lightweight interpreters and JITs
-
Running rustc in a browser
There has been discussion of doing this with MIRI, which would be easier than all of rustc.
-
Piecemeal dropping of struct members causes UB? (Miri)
This issue has been fixed: https://github.com/rust-lang/miri/issues/2964
- Erroneous UB Error with Miri?
-
I've incidentally created one of the fastest bounded MPSC queue
Actually, I've done more advanced tests with MIRI (see https://github.com/rust-lang/miri/issues/2920 for example) which allowed me to fix some issues. I've also made the code compatible with loom, but I didn't found the time yet to write and execute loom tests. That's on the TODO-list, and I need to track it with an issue too.
-
Interested in "secure programming languages", both theory and practice but mostly practice, where do I start?
He is one of the big brains behind Miri, which is a interpreter that runs on the MIR (compiler representation between human code and asm/machine code) and detects undefined behavior. Super useful tool for language safety, pretty interesting on its own.
-
Formal verification for unsafe code?
I would also run your tests in Miri (https://github.com/rust-lang/miri) to try to cover more bases.
-
Ouroboros is also unsound
You can run miri and it will tell you if the given run triggered any undefined behavior. It will not analyze it for every possible use of the code, but checking for the presence of this specific issue using it should be fairly simple.
What are some alternatives?
eioio - Effects-based direct-style IO for multicore OCaml
cons-list - Singly-linked list implementation in Rust
console - a debugger for async rust!
sanitizers - AddressSanitizer, ThreadSanitizer, MemorySanitizer
ocaml-multicore - Multicore OCaml
rust - Empowering everyone to build reliable and efficient software.
shuttle - Shuttle is a library for testing concurrent Rust code
Rust-Full-Stack - Rust projects here are easy to use. There are blog posts for them also.
TLAPLUS_DeadlockEmpire - Specs and models for solving the DeadlockEmpire problems using TLA+ and TLC
rfcs - RFCs for changes to Rust
triple-buffer - Implementation of triple buffering in Rust
nomicon - The Dark Arts of Advanced and Unsafe Rust Programming