mercury-ad
GPU-Puzzles
| mercury-ad | GPU-Puzzles | |
|---|---|---|
| 2 | 12 | |
| 6 | 5,074 | |
| - | - | |
| 10.0 | 3.4 | |
| over 1 year ago | 4 months ago | |
| Mercury | Jupyter Notebook | |
| GNU General Public License v3.0 only | MIT License |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
mercury-ad
-
Understanding Automatic Differentiation in 30 lines of Python
I wrote a purely functional AD library in Mercury [0], which adapts a general approach from [1]. I believe that Owl provides a similar approach [2].
[0] https://github.com/mclements/mercury-ad
-
Autodidax: Jax Core from Scratch (In Python)
I find the solutions from https://github.com/qobi/AD-Rosetta-Stone/ to be very helpful, particularly for representing forward and backward mode automatic differentiation using a functional approach.
I used this code as inspiration for a functional-only (without references/pointers) in Mercury: https://github.com/mclements/mercury-ad
GPU-Puzzles
- Solve Puzzles. Learn CUDA
- GPU Puzzles
- Understanding Automatic Differentiation in 30 lines of Python
-
FlashAttention-2, 2x faster than FlashAttention
I found it helpful to start with CUDA on numba since it lets you write GPU kernels in python. Assuming you're like most ML engineers and you're more familiar with python than C++, this allows you to separately learn CUDA concepts from also learning C++ at the same time. There's also a set of GPU puzzles for beginners [1] using to get started with numba CUDA.
[1] https://github.com/srush/GPU-Puzzles
- [Computer Science] srush/GPU-Puzzles: Solve puzzles. Learn CUDA.
-
Build on AWS Weekly - S1 E2 - Breaking Blocks with Terraform
Are you having fun with Machine Learning? Go and teach yourself beginner GPU programming with this wonderful notebook: GitHub repo
- GPU-Puzzles: Solve Puzzles. Learn CUDA
-
[D] What are some good resources to learn CUDA programming?
Practice puzzles: https://github.com/srush/GPU-Puzzles
- Learn GPU programming in interactive fashion
What are some alternatives?
AD-Rosetta-Stone - Examples of Automatic Differentiation (AD) in many different languages and systems
vscode-infracost - See cost estimates for Terraform right in your editor💰📉
autodidact - A pedagogical implementation of Autograd
triton - Development repository for the Triton language and compiler
autograd - Efficiently computes derivatives of numpy code.
cutlass - CUDA Templates for Linear Algebra Subroutines
owl - Owl - OCaml Scientific Computing @ https://ocaml.xyz
carbon-lang - Carbon Language's main repository: documents, design, implementation, and related tools. (NOTE: Carbon Language is experimental; see README)
terraform-minecraft - A Terraform Script that can deploy Minecraft Servers
Tensor-Puzzles - Solve puzzles. Improve your pytorch.
LLM-Training-Puzzles - What would you do with 1000 H100s...