jax VS tinygrad

localscope is a small Python package that disassembles functions to check if they access global variables they shouldn't. I wrote this a few years ago to detect scope bugs which are common in Jupyter notebooks. It's recently come in handy writing jax code (https://github.com/jax-ml/jax) because it requires pure functions. Thought I'd share.

If you like high-performance array programming a la "numpy with JIT" I suggest looking at JAX. It's very suitable for general numeric computing (not just ML) and a very mature ecosystem.

https://github.com/jax-ml/jax

Nope, changing graph shape requires recompilation: https://github.com/google/jax/discussions/17191

I found a GitHub issue that seemed similar (missing ptxas) and saw a suggestion to install nvidia-cuda-toolkit. Alright: but that exploded the container size from 6.5 GB to 12.13 GB … unacceptable 😤 (Incidentally, this is too large for Cloud Shell to build on its limited persistent disk.)

The dual numbers exist just as surely as the real numbers and have been used well over 100 years

https://en.m.wikipedia.org/wiki/Dual_number

Pytorch has had them for many years.

https://pytorch.org/docs/stable/generated/torch.autograd.for...

JAX implements them and uses them exactly as stated in this thread.

https://github.com/google/jax/discussions/10157#discussionco...

As you so eloquently stated, "you shouldn't be proclaiming things you don't actually know on a public forum," and doubly so when your claimed "corrections" are so demonstrably and totally incorrect.

On your last point, as long as you jit the topmost level, it doesn't matter whether or not you have inner jitted functions. The end result should be the same.

Source: https://github.com/google/jax/discussions/5199#discussioncom...

The design of MLX is inspired by frameworks like NumPy, PyTorch, Jax, and ArrayFire.

Thanks! That is a great question and one that I've we've been battling with as well. As far as we know, this is not possible due to the way different contexts are set up on the GPU https://github.com/pygfx/pygfx/issues/510

tinygrad which I haven't used seems torch-like and has a WGPU backend: https://github.com/tinygrad/tinygrad

I'm plugging away on my BitGrid project.... a Turing complete stripped down version of an FPGA without routing fabric and with added delays (for reasons). I'm learning KiCad 8.0, so I can do schematics and build a prototype cell out of TTL.[1]

I'm also re-acquainting myself with Verilog so I can do an ASIC prototype through TinyTapeout. The main question that I hope to answer is just how much power a bitgrid cell actually consumes, both static and dynamic. If it's low enough, then it'll give Petaflops to the masses, if not.. it's a curiosity.

Along that path, I've learned that the configuration memory for the LUTs is going to consume most of the silicon. Since it's all just D flip-flops... I figured I could dual-use it as memory without loss of generalization. You can virtually add 2 bytes of memory in a cell in any of the 4 directions... so I call it IsoLinear Memory.[2] ;-)

I should be able to make the deadline for TinyTapeout 9, in December. Meanwhile I'll update my emulator to include Isolinear Memory, and figure out how to program the damned thing. My stretch goal is to figure out how to program it from TinyGrad.[3].

If nothing else, it'll be good for real time DSP.

[1] https://github.com/mikewarot/BitGrid_TTL

[2] https://github.com/mikewarot/BitGrid_TTL/tree/master/IsoLine...

[3] https://github.com/tinygrad/tinygrad

Umm, why not?

We wrote entire NVIDIA, AMD, and QCOM drivers in that style.

https://github.com/tinygrad/tinygrad/blob/master/tinygrad/ru...

https://github.com/tinygrad/tinygrad/blob/master/tinygrad/ru...

https://github.com/tinygrad/tinygrad/blob/master/tinygrad/ru...

He seems to be active: https://github.com/tinygrad/tinygrad/commits?author=geohot

Portability is a priority, as machine learning frameworks are often too large to be included in prebuilt installers. However, emerging frameworks like tinygrad offer smaller sizes that can be reasonably included, making it easier for less technical users to install. I'm planning to eventually switch all inference tasks to one framework, which will further simplify the installation process.️

Tinygrad offers a superior MNIST implementation with minimal dependencies[0].

[0] https://github.com/tinygrad/tinygrad/blob/master/docs/mnist....

I have not been impressed by the perf. Slower than PyTorch for LLMs, and PyTorch is actually stable on AMD (I've trained 7B/13B models).. so the stability issues seem to be more of a tinygrad problem and less of an AMD problem, despite George's ramblings [0][1]

[0] https://github.com/tinygrad/tinygrad/issues/4301