FastFold
TransformerEngine
| FastFold | TransformerEngine | |
|---|---|---|
| 2 | 2 | |
| 506 | 1,438 | |
| - | 5.0% | |
| 0.0 | 9.5 | |
| 10 months ago | about 12 hours ago | |
| Python | Python | |
| Apache License 2.0 | Apache License 2.0 |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
FastFold
-
👉 Impressed With AlphaFold? Checkout This Protein Structure Prediction Model (FastFold) That Reduces AlphaFold’s Training Time From 11 Days To 67 Hours
Code for https://arxiv.org/abs/2203.00854 found: https://github.com/hpcaitech/FastFold
Github: https://github.com/hpcaitech/FastFold
TransformerEngine
-
Benchmarking Large Language Models on NVIDIA H100 GPUs with CoreWeave (Part 1)
4090 now has its 8-bit float enabled as well, see the [transformer engine issue](https://github.com/NVIDIA/TransformerEngine/issues/15)
-
GPUs for Deep Learning in 2023 – An In-depth Analysis
Would be curious to see your benchmarks. Btw, Nvidia will be providing support for fp8 in a future release of CUDA - https://github.com/NVIDIA/TransformerEngine/issues/15
I think TMA may not matter as much for consumer cards given the disproportionate amount of fp32 / int32 compute that they have.
Would be interesting to see how close to theoretical folks are able to get once CUDA support comes through.
What are some alternatives?
torchsynth - A GPU-optional modular synthesizer in pytorch, 16200x faster than realtime, for audio ML researchers.
Whisper - High-performance GPGPU inference of OpenAI's Whisper automatic speech recognition (ASR) model
openfold - Trainable, memory-efficient, and GPU-friendly PyTorch reproduction of AlphaFold 2
autocvd - Tool to automatically set CUDA_VISIBLE_DEVICES based on GPU utilization. Usable from command line and code.
warp-drive - Extremely Fast End-to-End Deep Multi-Agent Reinforcement Learning Framework on a GPU (JMLR 2022)
torchrec - Pytorch domain library for recommendation systems
ivy - The Unified AI Framework
nanoGPT - The simplest, fastest repository for training/finetuning medium-sized GPTs.
fastaudio - 🔊 Audio and fastai v2
liberate-fhe - A Fully Homomorphic Encryption (FHE) library for bridging the gap between theory and practice with a focus on performance and accuracy.