Pytorch VS tinygrad

TensorFlow

TensorFlow, developed by Google, and PyTorch, developed by Facebook, are two of the most popular frameworks for building and training complex machine learning models. TensorFlow is known for its flexibility and robust scalability, making it suitable for both research prototypes and production deployments. PyTorch is praised for its ease of use, simplicity, and dynamic computational graph that allows for more intuitive coding of complex AI models. Both frameworks support a wide range of AI models, from simple linear regression to complex deep neural networks.

> does PyTorch have a similar concept

of course https://github.com/pytorch/pytorch/blob/main/torch/utils/_py...

fyi should work on most 40xx[1]

[1] https://github.com/pytorch/pytorch/issues/119638#issuecommen...

Sure, right here: https://github.com/pytorch/pytorch/blob/main/torch/autograd/...

Here's the documentation: https://pytorch.org/tutorials/intermediate/forward_ad_usage....

> When an input, which we call “primal”, is associated with a “direction” tensor, which we call “tangent”, the resultant new tensor object is called a “dual tensor” for its connection to dual numbers[0].

*My post explains Dot, Matrix and Element-wise multiplication in PyTorch.

In PyTorch with @, dot() or matmul():

import torch # we use PyTorch: https://pytorch.org data = torch.tensor(encode(text), dtype=torch.long) print(data.shape, data.dtype) print(data[:1000]) # the 1000 characters we looked at earlier will to the GPT look like this

AI's Open Embrace Artificial intelligence (AI) and machine learning (ML) are increasingly leveraging open-source frameworks like TensorFlow [https://www.tensorflow.org/] and PyTorch [https://pytorch.org/]. This democratization of AI tools is driving innovation and lowering entry barriers across industries.

Which label applies to a tool sometimes depends on what you do with it. For example, PyTorch or TensorFlow can be called a library, a toolkit, or a machine-learning framework.

Might be a silly question but is GGML a similar/competing library to George Hotz's tinygrad [0]?

[0] https://github.com/geohot/tinygrad

at first glance i thought may its like tinygrad. but looks has many ops than that tiny grad but most maps to underlying hardware provided ops?

i wonder how well tinygrad's apporach will work out, ops fusion sounds easy, just a walk a graph, pattern match it and lower to hardware provided ops?

Anyway if anyone wants to understand the philosophy behind tinygrad, this file is great start https://github.com/geohot/tinygrad/blob/master/docs/abstract...

There are currently at least 3 ways to run llama on m1 with GPU acceleration. - mlc-llm (pre-built, only 1 model has been ported) - tinygrad (very memory efficient, not that easy to integrate into other projects) - llama-mps (original llama codebase + llama adapter support)

Hopefully we will see now full support with AMD hardware on https://github.com/geohot/tinygrad. You can read more about it on https://tinygrad.org/

tinygrad

https://github.com/geohot/tinygrad/tree/master/accel/ane

But I have not tested it on Linux since Asahi has not yet added support.

llama.cpp runs at 18ms per token (7B) and 200ms per token (65B) without quantization.