Pytorch VS tinygrad

Python’s Growth in Data Work and AI: Python continues to lead because of its easy-to-read style and the huge number of libraries available for tasks from data work to artificial intelligence. Tools like TensorFlow and PyTorch make it a must-have. Whether you’re experienced or just starting, Python’s clear style makes it a good choice for diving into machine learning. Actionable Tip: If you’re new to Python, try projects that combine data with everyday problems. For example, build a simple recommendation system using Pandas and scikit-learn.

Basically this style of code - https://github.com/pytorch-labs/attention-gym/pull/84/files - has issues like this - https://github.com/pytorch/pytorch/pull/137452 https://github.com/pytorch/pytorch/issues/144511 https://github.com/pytorch/pytorch/issues/145869

For some higher level context, see https://pytorch.org/blog/flexattention/

The AI Service will be built using aiohttp (asynchronous Python web server) and integrates PyTorch, Hugging Face Transformers, numpy, pandas, and scikit-learn for financial data analysis.

Open source frameworks like PyTorch are already enabling Machine Learning breakthroughs because they’re living communities where great things happen through:

- Data Science and AI: TensorFlow, PyTorch and scikit-learn are only a few of the standard Python libraries. - Web Development: development of web-based applications is made simple by frameworks such as Flask as well as Django. - Prototyping: Python's ease of use lets you quickly iterate and testing concepts.

By chance, Tensorflow or PyTorch can work with pip packages from Nvidia.

PyTorch

> It's quite common in machine learning operations to multiply a matrix of unsigned byte by a matrix of signed byte. Don't ask me why, but that's the case.

Overflow is the reason. Intel's vpmaddubsw takes int8_t and uint8_t to give you results in int16_t. If both are unsigned 255 * 255 = 65025 will be out of range for int16_t so likely the instruction is designed to take int8_t and uint8_t. The overflow (or rather saturation with this instruction) can still occur because it sums to adjacent multiplication. See my comment in PyTorch. https://github.com/pytorch/pytorch/blob/a37db5ae3978010e1bb7...

I'm plugging away on my BitGrid project.... a Turing complete stripped down version of an FPGA without routing fabric and with added delays (for reasons). I'm learning KiCad 8.0, so I can do schematics and build a prototype cell out of TTL.[1]

I'm also re-acquainting myself with Verilog so I can do an ASIC prototype through TinyTapeout. The main question that I hope to answer is just how much power a bitgrid cell actually consumes, both static and dynamic. If it's low enough, then it'll give Petaflops to the masses, if not.. it's a curiosity.

Along that path, I've learned that the configuration memory for the LUTs is going to consume most of the silicon. Since it's all just D flip-flops... I figured I could dual-use it as memory without loss of generalization. You can virtually add 2 bytes of memory in a cell in any of the 4 directions... so I call it IsoLinear Memory.[2] ;-)

I should be able to make the deadline for TinyTapeout 9, in December. Meanwhile I'll update my emulator to include Isolinear Memory, and figure out how to program the damned thing. My stretch goal is to figure out how to program it from TinyGrad.[3].

If nothing else, it'll be good for real time DSP.

[1] https://github.com/mikewarot/BitGrid_TTL

[2] https://github.com/mikewarot/BitGrid_TTL/tree/master/IsoLine...

[3] https://github.com/tinygrad/tinygrad

Umm, why not?

We wrote entire NVIDIA, AMD, and QCOM drivers in that style.

https://github.com/tinygrad/tinygrad/blob/master/tinygrad/ru...

https://github.com/tinygrad/tinygrad/blob/master/tinygrad/ru...

https://github.com/tinygrad/tinygrad/blob/master/tinygrad/ru...

He seems to be active: https://github.com/tinygrad/tinygrad/commits?author=geohot

Portability is a priority, as machine learning frameworks are often too large to be included in prebuilt installers. However, emerging frameworks like tinygrad offer smaller sizes that can be reasonably included, making it easier for less technical users to install. I'm planning to eventually switch all inference tasks to one framework, which will further simplify the installation process.️

Tinygrad offers a superior MNIST implementation with minimal dependencies[0].

[0] https://github.com/tinygrad/tinygrad/blob/master/docs/mnist....

I have not been impressed by the perf. Slower than PyTorch for LLMs, and PyTorch is actually stable on AMD (I've trained 7B/13B models).. so the stability issues seem to be more of a tinygrad problem and less of an AMD problem, despite George's ramblings [0][1]

[0] https://github.com/tinygrad/tinygrad/issues/4301