Notes on Concurrency Bugs

This page summarizes the projects mentioned and recommended in the original post on news.ycombinator.com

Our great sponsors
  • InfluxDB - Power Real-Time Data Analytics at Scale
  • WorkOS - The modern identity platform for B2B SaaS
  • SaaSHub - Software Alternatives and Reviews
  • triple-buffer

    Implementation of triple buffering in Rust

  • In terms of shared-memory threading concurrency, Send and Sync, and the distinction between &T and &Mutex and &mut T, were a revelation when I first learned them. It was a principled approach to shared-memory threading, with Send/Sync banning nearly all of the confusing and buggy entangled-state codebases I've seen and continue to see in C++ (much to my frustration and exasperation), and &Mutex providing a cleaner alternative design (there's an excellent article on its design at http://cliffle.com/blog/rust-mutexes/).

    My favorite simple concurrent data structure is https://docs.rs/triple_buffer/latest/triple_buffer/struct.Tr.... It beautifully demonstrates how you can achieve principled shared mutability, by defining two "handle" types (living on different threads), each carrying thread-local state (not TLS) and a pointer to shared memory, and only allowing each handle to access shared memory in a particular way. This statically prevents one thread from calling a method intended to run on another thread, or accessing fields local to another thread (since the methods and fields now live on the other handle). It also demonstrates the complexity of reasoning about lock-free algorithms (https://github.com/HadrienG2/triple-buffer/issues/14).

    I suppose &/&mut is also a safeguard against event-loop and reentrancy bugs (like https://github.com/quotient-im/Quaternion/issues/702). I don't think Rust solves the general problem of preventing deadlocks within and between processes (which often cross organizational boundaries between projects and distinct codebases, with no clear contract on allowed behavior and which party in a deadlock is at fault), and non-atomicity between processes on a single machine (see my PipeWire criticism at https://news.ycombinator.com/item?id=31519951). File saving is also difficult (https://danluu.com/file-consistency/), though I find that fsync-then-rename works well enough if you don't need to preserve metadata or write through file (not folder) symlinks.

  • Quaternion

    A Qt-based IM client for Matrix

  • In terms of shared-memory threading concurrency, Send and Sync, and the distinction between &T and &Mutex and &mut T, were a revelation when I first learned them. It was a principled approach to shared-memory threading, with Send/Sync banning nearly all of the confusing and buggy entangled-state codebases I've seen and continue to see in C++ (much to my frustration and exasperation), and &Mutex providing a cleaner alternative design (there's an excellent article on its design at http://cliffle.com/blog/rust-mutexes/).

    My favorite simple concurrent data structure is https://docs.rs/triple_buffer/latest/triple_buffer/struct.Tr.... It beautifully demonstrates how you can achieve principled shared mutability, by defining two "handle" types (living on different threads), each carrying thread-local state (not TLS) and a pointer to shared memory, and only allowing each handle to access shared memory in a particular way. This statically prevents one thread from calling a method intended to run on another thread, or accessing fields local to another thread (since the methods and fields now live on the other handle). It also demonstrates the complexity of reasoning about lock-free algorithms (https://github.com/HadrienG2/triple-buffer/issues/14).

    I suppose &/&mut is also a safeguard against event-loop and reentrancy bugs (like https://github.com/quotient-im/Quaternion/issues/702). I don't think Rust solves the general problem of preventing deadlocks within and between processes (which often cross organizational boundaries between projects and distinct codebases, with no clear contract on allowed behavior and which party in a deadlock is at fault), and non-atomicity between processes on a single machine (see my PipeWire criticism at https://news.ycombinator.com/item?id=31519951). File saving is also difficult (https://danluu.com/file-consistency/), though I find that fsync-then-rename works well enough if you don't need to preserve metadata or write through file (not folder) symlinks.

  • InfluxDB

    Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

    InfluxDB logo
  • SPSCQueue.h

    A bounded single-producer single-consumer wait-free and lock-free queue written in C++11

  • A triple buffer is a good choice if all you want is polling the latest data at any given time, and you want to avoid mutexes altogether. If you want each piece of data to be delivered exactly once, you can use a queue (bounded or "unlimited" though the latter doesn't supply backpressure which I hear causes problems). SPSC lock-free bounded queues are dead simple to write, and can be tuned for higher throughput even with contention (https://github.com/rigtorp/SPSCQueue claims to be nice, and I haven't had issues working with it aside from having to peek and pop separately, but it's C++, and not a misuse-proof API since it doesn't use the "handles" idea I talked about, and you can push/read/pop from the wrong thread). If you want the reader to poll/WaitForMultipleObjects until the queue has items, that has to be done separately from the SPSC.

    And mutexes make a lot of things easier... and introduces "oops wrong mutex!" (Rust solves it) and deadlock (Rust doesn't solve it).

NOTE: The number of mentions on this list indicates mentions on common posts plus user suggested alternatives. Hence, a higher number means a more popular project.

Suggest a related project

Related posts