Pytorch VS CPython

Hi everyone! I’m devloker, and today I’m excited to share a project I’ve been working on: a digit recognition system implemented using pure math functions in Python. This project aims to help beginners grasp the mathematics behind AI and digit recognition without relying on high-level libraries like TensorFlow or PyTorch. You can find the complete code on my GitHub repository.

PyTorch

> their own custom stack to interact with GPUs

lol completely made up.

are you conflating CUDA the platform with the C/C++ like language that people write into files that end with .cu? because while some people are indeed not writing .cu files, absolutely no one is skipping the rest of the "stack".

source: i work at one of these "mega corps". hell if you don't believe me go look at how many CUDA kernels pytorch has https://github.com/pytorch/pytorch/tree/main/aten/src/ATen/n....

> Everybody thinks it’s CUDA that makes Nvidia the dominant player.

it 100% does

PyTorch - An open source machine learning framework. PyTorch Tutorials - Tutorials and documentation.

In this guide, we provided a comprehensive, step-by-step explanation of how to implement a simple GPT (Generative Pre-trained Transformer) model using PyTorch. We walked through the process of creating a custom dataset, building the GPT model, training it, and generating text. This hands-on implementation demonstrates the fundamental concepts behind the GPT architecture and serves as a foundation for more complex applications. By following this guide, you now have a basic understanding of how to create, train, and utilize a simple GPT model. This knowledge equips you to experiment with different configurations, larger datasets, and additional techniques to enhance the model's performance and capabilities. The principles and techniques covered here will help you apply transformer models to various NLP tasks, unlocking the potential of deep learning in natural language understanding and generation. The methodologies presented align with the advancements in transformer models introduced by Vaswani et al. (2017), emphasizing the power of self-attention mechanisms in processing sequences of data more effectively than traditional approaches (Vaswani et al., 2017). This understanding opens pathways to explore and innovate in the field of natural language processing using cutting-edge deep learning techniques (Kingma & Ba, 2015).

PyTorch is a powerful and flexible deep learning framework that offers a rich set of features for building and training neural networks.

Oddly enough, sometimes, the best way to learn is by putting forth incorrect opinions or questions. Recently, while wrestling with AI project complexities, I pondered aloud whether all Docker images with AI models would inevitably be bulky due to PyTorch dependencies. To my surprise, this sparked many helpful responses, offering insights into optimizing image sizes. Being willing to be wrong opens up avenues for rapid learning.

Tinygrad targets consumer hardware (to be precise, only Radeon 7900XTX and nothing else[1]), while ROCm does not actually provide good support for such hardware. For example, last release of hipBLASLt-6.1.1 library has deep integration with PyTorch[1], while working only on AMD Instinct hardware. And even for the professional hardware out there, the support period is ridiculous: AMD Instinct MI100 (2020) is not supported. Only 4 years and tens of thousands of dollars worth of hardware is going to the trash, yay!

And to be more precise, they still use some core libraries from ROCm stack[3], they just don't use all these fancy multi-gigabyte[4] hardware-limited rocBLAS/hipBLASlt/rocWMMA/rocRAND/etc. libraries.

[1] https://tinygrad.org/#tinybox

[2] https://github.com/pytorch/pytorch/issues/119081

[3] https://github.com/tinygrad/tinygrad/blob/v0.9.0/tinygrad/ru...

[4] https://repo.radeon.com/rocm/yum/6.1.1/main/

PyTorch (https://pytorch.org/)

Imagine planning your dream vacation, but instead of spending hours searching online, an intelligent assistant arranges everything for you. Sounds like a fantasy, right? What if I said you could build this AI travel agent with just a few lines of Python code? Yes, it's that simple. In this guide, we'll walk you through the process step-by-step, so you can have your travel assistant ready to help with your next adventure.

python has random.shuffle() and random.sample() with an MT Mersenne Twister PRNG for random. https://docs.python.org/3/library/random.html#random.shuffle Modules/_randommodule.c: https://github.com/python/cpython/blob/main/Modules/_randomm... , Library/random.py: https://github.com/python/cpython/blob/main/Lib/random.py#L3...

From "Uniting the Linux random-number devices" (2022) https://news.ycombinator.com/item?id=30377944 :

> > In 2020, the Linux kernel version 5.6 /dev/random only blocks when the CPRNG hasn't initialized. Once initialized, /dev/random and /dev/urandom behave the same. [17]

From https://news.ycombinator.com/item?id=37712506 :

> "lock-free concurrency" [...] "Ask HN: Why don't PCs have better entropy sources?" [for generating txids/uuids] https://news.ycombinator.com/item?id=30877296

> "100-Gbit/s Integrated Quantum Random Number Generator Based on Vacuum Fluctuations" https://link.aps.org/doi/10.1103/PRXQuantum.4.010330

Python

Python bytecode is like a secret language that Python uses behind the scenes. When you write your Python code, it doesn’t run directly. Instead, Python translates your code into bytecode, a set of instructions that the Python interpreter can understand and execute.

Ensure you have Python installed on your system. You can download it from the official Python website. Verify the installation by running:

We strongly encourage maintainers of third-party Python projects to test with 3.13 during the beta phase and report issues found to the Python bug tracker as soon as possible. While the release is planned to be feature complete entering the beta phase, it is possible that features may be modified or, in rare cases, deleted up until the start of the release candidate phase (Tuesday 2024-07-30). Our goal is to have no ABI changes after beta 4 and as few code changes as possible after 3.13.0rc1, the first release candidate. To achieve that, it will be extremely important to get as much exposure for 3.13 as possible during the beta phase.

Before diving into Tkinter, ensure that you have Python installed on your computer. Visit the official Python website (https://www.python.org) and download the latest version suitable for your operating system.

1. Download the files: "CasinoGame.py" and "MainDriverSection.py" 2. Make sure Python is installed: https://www.python.org/ 3. Open a terminal for command prompt 3.1 Navigate to the directory containing the files 3.2 Run the game with "python3

Let's install Python on your computer by downloading it from the official Python website.