micrograd VS Rocket

I recently decided to turbo-teach myself basic cuda with a proper project. I really enjoyed Karpathy’s micrograd (https://github.com/karpathy/micrograd), so I extended it with cuda kernels and 2D tensor logic. It’s a bit longer than the original project, but it’s still very readable for anyone wanting to quickly learn about gpu acceleration in practice.

FOr inference, less than 1KLOC of pure, dependency-free C is enough (if you include the tokenizer and command line parsing)[1]. This was a non-obvious fact for me, in principle, you could run a modern LLM 20 years ago with just 1000 lines of code, assuming you're fine with things potentially taking days to run of course.

Training wouldn't be that much harder, Micrograd[2] is 200LOC of pure Python, 1000 lines would probably be enough for training an (extremely slow) LLM. By "extremely slow", I mean that a training run that normally takes hours could probably take dozens of years, but the results would, in principle, be the same.

If you were writing in C instead of Python and used something like Llama CPP's optimization tricks, you could probably get somewhat acceptable training performance in 2 or 3 KLOC. You'd still be off by one or two orders of magnitude when compared to a GPU cluster, but a lot better than naive, loopy Python.

[1] https://github.com/karpathy/llama2.c

[2] https://github.com/karpathy/micrograd

Perhaps they were thinking of https://github.com/karpathy/micrograd

I was (for fun) trying to recreate micrograd in Haskell. The ideia is simple:

Check out this repo I found a few years back when I was looking into understanding pytorch better. It's basically a super tiny autodiff library that only works on scalars. The whole repo is under 200 lines of code, so you can pull up pycharm or whatever and step through the code and see how it all comes together. Or... you know. Just read it, it's not super complicated.

I'm doing an ML apprenticeship [1] these weeks and Karpathy's videos are part of it. We've been deep down into them. I found them excellent. All concepts he illustrates are crystal clear in his mind (even though they are complicated concepts themselves) and that shows in his explanations.

Also, the way he builds up everything is magnificent. Starting from basic python classes, to derivatives and gradient descent, to micrograd [2] and then from a bigram counting model [3] to makemore [4] and nanoGPT [5]

[1]: https://www.foundersandcoders.com/ml

[2]: https://github.com/karpathy/micrograd

[3]: https://github.com/karpathy/randomfun/blob/master/lectures/m...

[4]: https://github.com/karpathy/makemore

[5]: https://github.com/karpathy/nanoGPT

Just published my first crate, rustygrad, a Rust implementation of Andrej Karpathy's micrograd!

I've been trying to reimplement Karpathy's micrograd library in rust as a fun side project.

4. Rocket

If you want something simpler/more minimal, you could use https://rocket.rs/ for the backend and handle the front-end however you want.

I’m the author of https://kellnr.io. When I started working on Kellnr three years ago, https://rocket.rs was “the web framework” to use. Unfortunately, the project seems dead. Before adding more functionality using an unmaintained framework, I want to port Kellnr to https://github.com/tokio-rs/axum.

Or else you could of course just use https://rocket.rs/

Rust is a popular system programming language, known for its robust memory safety features and exceptional performance. While Rust was originally a system programming language, its application has evolved. Now you can see Rust in different app platforms, mobile apps, and of course, in web apps — both in the frontend and backend, with frameworks like Rocket, Axum, and Actix making it even easier to build web applications with Rust.

I've done some research but I have to admit that creating embedded devices is a totally new subject for me, but that is the point of the project - main goal is learning, and creating something is the secondary goal, so please bear with me and my knowledge of the subject. So, for the hardware I've seen many people recommending SMT32 family devices, but I've also read that anything with the Cortex-M processor can be suitable. Need more info on that. OS is a hard choice for me because on one hand I was thinking of Ubuntu Core but the device support is not really that good I think, so other options I've found are Tock and RIOT-OS, and I am gravitating towards the latter because it's main focus is on IOT devices. I've found frameworks like Rocket.rs for a web app, tauri.app for desktop app (which might not be needed but I still like the idea). Also found Tokio.rs which apparently will help with the networking. There was a discussion from the other members about using the Golioth cloud platform with Zephyr and C++, and I don't know if there are any other alternatives for Golioth that support Rust, I've found webthings.io but I am not sure if it's an alternative, or something else actually, so I would be happy to learn more about that. Again I want to hear your recommendations regarding anything that will help creating a project like that.

I use Rocket on the backend with Postgres. Currently experimenting with Yew for the frontend.