SPSCQueue.h
NCCL
SPSCQueue.h | NCCL | |
---|---|---|
1 | 3 | |
829 | 2,825 | |
- | 2.2% | |
4.6 | 5.8 | |
4 months ago | 7 days ago | |
C++ | C++ | |
MIT License | GNU General Public License v3.0 or later |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
SPSCQueue.h
-
Notes on Concurrency Bugs
A triple buffer is a good choice if all you want is polling the latest data at any given time, and you want to avoid mutexes altogether. If you want each piece of data to be delivered exactly once, you can use a queue (bounded or "unlimited" though the latter doesn't supply backpressure which I hear causes problems). SPSC lock-free bounded queues are dead simple to write, and can be tuned for higher throughput even with contention (https://github.com/rigtorp/SPSCQueue claims to be nice, and I haven't had issues working with it aside from having to peek and pop separately, but it's C++, and not a misuse-proof API since it doesn't use the "handles" idea I talked about, and you can push/read/pop from the wrong thread). If you want the reader to poll/WaitForMultipleObjects until the queue has items, that has to be done separately from the SPSC.
And mutexes make a lot of things easier... and introduces "oops wrong mutex!" (Rust solves it) and deadlock (Rust doesn't solve it).
NCCL
-
MPI jobs to test
% rm -rf /tmp/nccl ; git clone --recursive https://github.com/NVIDIA/nccl.git ; cd nccl ; git grep MPI Cloning into 'nccl'... remote: Enumerating objects: 2769, done. remote: Counting objects: 100% (336/336), done. remote: Compressing objects: 100% (140/140), done. remote: Total 2769 (delta 201), reused 287 (delta 196), pack-reused 2433 Receiving objects: 100% (2769/2769), 3.04 MiB | 3.37 MiB/s, done. Resolving deltas: 100% (1820/1820), done. README.md:NCCL (pronounced "Nickel") is a stand-alone library of standard communication routines for GPUs, implementing all-reduce, all-gather, reduce, broadcast, reduce-scatter, as well as any send/receive based communication pattern. It has been optimized to achieve high bandwidth on platforms using PCIe, NVLink, NVswitch, as well as networking using InfiniBand Verbs or TCP/IP sockets. NCCL supports an arbitrary number of GPUs installed in a single node or across multiple nodes, and can be used in either single- or multi-process (e.g., MPI) applications. src/collectives/broadcast.cc:/* Deprecated original "in place" function, similar to MPI */
-
NVLink and Dual 3090s
If it's rendering, you don't really need SLI, you need to install NCCL so that GPUs memory can be pooled: https://github.com/NVIDIA/nccl
-
Distributed Training Made Easy with PyTorch-Ignite
backends from native torch distributed configuration: nccl, gloo, mpi.
What are some alternatives?
moodycamel - A fast multi-producer, multi-consumer lock-free concurrent queue for C++11
gloo - Collective communications library with various primitives for multi-machine training.
HPX - The C++ Standard Library for Parallelism and Concurrency
C++ Actor Framework - An Open Source Implementation of the Actor Model in C++
libdill - Structured concurrency in C
Thrust - [ARCHIVED] The C++ parallel algorithms library. See https://github.com/NVIDIA/cccl
VexCL - VexCL is a C++ vector expression template library for OpenCL/CUDA/OpenMP
Taskflow - A General-purpose Parallel and Heterogeneous Task Programming System
xla - Enabling PyTorch on XLA Devices (e.g. Google TPU)
libcds - A C++ library of Concurrent Data Structures
Easy Creation of GnuPlot Scripts from C++ - A simple C++17 lib that helps you to quickly plot your data with GnuPlot