NCCL
ignite
NCCL | ignite | |
---|---|---|
3 | 3 | |
2,825 | 4,458 | |
2.2% | 0.4% | |
5.8 | 8.7 | |
9 days ago | 6 days ago | |
C++ | Python | |
GNU General Public License v3.0 or later | BSD 3-clause "New" or "Revised" License |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
NCCL
-
MPI jobs to test
% rm -rf /tmp/nccl ; git clone --recursive https://github.com/NVIDIA/nccl.git ; cd nccl ; git grep MPI Cloning into 'nccl'... remote: Enumerating objects: 2769, done. remote: Counting objects: 100% (336/336), done. remote: Compressing objects: 100% (140/140), done. remote: Total 2769 (delta 201), reused 287 (delta 196), pack-reused 2433 Receiving objects: 100% (2769/2769), 3.04 MiB | 3.37 MiB/s, done. Resolving deltas: 100% (1820/1820), done. README.md:NCCL (pronounced "Nickel") is a stand-alone library of standard communication routines for GPUs, implementing all-reduce, all-gather, reduce, broadcast, reduce-scatter, as well as any send/receive based communication pattern. It has been optimized to achieve high bandwidth on platforms using PCIe, NVLink, NVswitch, as well as networking using InfiniBand Verbs or TCP/IP sockets. NCCL supports an arbitrary number of GPUs installed in a single node or across multiple nodes, and can be used in either single- or multi-process (e.g., MPI) applications. src/collectives/broadcast.cc:/* Deprecated original "in place" function, similar to MPI */
-
NVLink and Dual 3090s
If it's rendering, you don't really need SLI, you need to install NCCL so that GPUs memory can be pooled: https://github.com/NVIDIA/nccl
-
Distributed Training Made Easy with PyTorch-Ignite
backends from native torch distributed configuration: nccl, gloo, mpi.
ignite
-
Introducing PyTorch-Ignite's Code Generator v0.2.0
Along with the PyTorch-Ignite 0.4.5 release, we are excited to announce the new release of the web application for generating PyTorch-Ignite's training pipelines. This blog post is an overview of the key features and updates of the Code Generator v0.2.0 project release.
-
Distributed Training Made Easy with PyTorch-Ignite
PyTorch-Ignite's ignite.distributed (idist) submodule introduced in version v0.4.0 (July 2020) quickly turns single-process code into its data distributed version.
-
Introduction to PyTorch-Ignite
More details about distributed helpers provided by PyTorch-Ignite can be found in the documentation. A complete example of training on CIFAR10 can be found here.
What are some alternatives?
gloo - Collective communications library with various primitives for multi-machine training.
torch-metrics - Metrics for model evaluation in pytorch
C++ Actor Framework - An Open Source Implementation of the Actor Model in C++
image-similarity-measures - :chart_with_upwards_trend: Implementation of eight evaluation metrics to access the similarity between two images. The eight metrics are as follows: RMSE, PSNR, SSIM, ISSM, FSIM, SRE, SAM, and UIQ.
Thrust - [ARCHIVED] The C++ parallel algorithms library. See https://github.com/NVIDIA/cccl
prometheus_flask_exporter - Prometheus exporter for Flask applications
HPX - The C++ Standard Library for Parallelism and Concurrency
pymetrix - A simple Plug and Play Library for getting analytics. See website for docs.
xla - Enabling PyTorch on XLA Devices (e.g. Google TPU)
Easy Creation of GnuPlot Scripts from C++ - A simple C++17 lib that helps you to quickly plot your data with GnuPlot
code-generator - Web Application to generate your training scripts with PyTorch Ignite