micrograd
Exercism - Scala Exercises
Our great sponsors
micrograd | Exercism - Scala Exercises | |
---|---|---|
22 | 399 | |
8,273 | 7,265 | |
- | 0.4% | |
0.0 | 3.5 | |
5 days ago | about 2 months ago | |
Jupyter Notebook | ||
MIT License | - |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
micrograd
-
Micrograd-CUDA: adapting Karpathy's tiny autodiff engine for GPU acceleration
I recently decided to turbo-teach myself basic cuda with a proper project. I really enjoyed Karpathy’s micrograd (https://github.com/karpathy/micrograd), so I extended it with cuda kernels and 2D tensor logic. It’s a bit longer than the original project, but it’s still very readable for anyone wanting to quickly learn about gpu acceleration in practice.
-
Stuff we figured out about AI in 2023
FOr inference, less than 1KLOC of pure, dependency-free C is enough (if you include the tokenizer and command line parsing)[1]. This was a non-obvious fact for me, in principle, you could run a modern LLM 20 years ago with just 1000 lines of code, assuming you're fine with things potentially taking days to run of course.
Training wouldn't be that much harder, Micrograd[2] is 200LOC of pure Python, 1000 lines would probably be enough for training an (extremely slow) LLM. By "extremely slow", I mean that a training run that normally takes hours could probably take dozens of years, but the results would, in principle, be the same.
If you were writing in C instead of Python and used something like Llama CPP's optimization tricks, you could probably get somewhat acceptable training performance in 2 or 3 KLOC. You'd still be off by one or two orders of magnitude when compared to a GPU cluster, but a lot better than naive, loopy Python.
[1] https://github.com/karpathy/llama2.c
[2] https://github.com/karpathy/micrograd
-
Writing a C compiler in 500 lines of Python
Perhaps they were thinking of https://github.com/karpathy/micrograd
- Linear Algebra for Programmers
- Understanding Automatic Differentiation in 30 lines of Python
-
Newbie question: Is there overloading of Haskell function signature?
I was (for fun) trying to recreate micrograd in Haskell. The ideia is simple:
-
[D] Backpropagation is not just the chain-rule, then what is it?
Check out this repo I found a few years back when I was looking into understanding pytorch better. It's basically a super tiny autodiff library that only works on scalars. The whole repo is under 200 lines of code, so you can pull up pycharm or whatever and step through the code and see how it all comes together. Or... you know. Just read it, it's not super complicated.
-
Neural Networks: Zero to Hero
I'm doing an ML apprenticeship [1] these weeks and Karpathy's videos are part of it. We've been deep down into them. I found them excellent. All concepts he illustrates are crystal clear in his mind (even though they are complicated concepts themselves) and that shows in his explanations.
Also, the way he builds up everything is magnificent. Starting from basic python classes, to derivatives and gradient descent, to micrograd [2] and then from a bigram counting model [3] to makemore [4] and nanoGPT [5]
[1]: https://www.foundersandcoders.com/ml
[2]: https://github.com/karpathy/micrograd
[3]: https://github.com/karpathy/randomfun/blob/master/lectures/m...
[4]: https://github.com/karpathy/makemore
[5]: https://github.com/karpathy/nanoGPT
-
Rustygrad - A tiny Autograd engine inspired by micrograd
Just published my first crate, rustygrad, a Rust implementation of Andrej Karpathy's micrograd!
-
Hey Rustaceans! Got a question? Ask here (10/2023)!
I've been trying to reimplement Karpathy's micrograd library in rust as a fun side project.
Exercism - Scala Exercises
-
5 Websites to Boost Your Coding and Master Algorithms 🚀
Exercism
-
MDN Curriculum
Nice, this reminds me of Exercism, which I wish was more widely known since they seem to be good folks. (disclaimer, I donate to them)
https://exercism.org/
-
Do 48 Programming Challenges in 2024 #48in24
Exercism, the free programming learning platform has initiated a challenge named: 48in24.
-
I learned* 12 languages in 2023: a retrospective
Last year, Exercism put together the #12in23 challenge. The goal was to learn a new programming language each month throughout the year. I was one of 135 people who completed the challenge, and I learned a lot along the way!
-
12in24 - One language a month
The list of languages contains every language on Exercism, excluding ones that I've used before, web languages, or ones that I can't download for some reason.
-
Ask HN: Programming Courses for Experienced Coders?
You might like https://exercism.org/
Learning by doing, with the help of mentors. Excellent way to learn a next language (as you are already familiar with the programming concepts).
- Any programs or websites to practice programming?
-
Best platform for coding & programming testing everyday to improve coding skills in various language?
Exercism is pretty good for beginners with some programming language, they are open source and worth contributing to.
-
Best Codewars for practice which have reflection in Web-Dev job.
Exercism
-
Show HN: Open-source tool for creating courses like Duolingo
> it might be more sustainable if courses were stored in a version controllable medium to facilitate multiple collaborators
My initial thought was to actually use GitHub to store the content. Either on Markdown or JSON - to have some version control. I like how Exercism [1] does it. But I thought it would be hard for teachers - unfamiliar with Git - to update lessons.
Then, I thought about implementing a version control system for the project but I felt I was overcomplicating things for an MVP. But I like the idea of having some kind of version control to improve collaboration.
[1] https://exercism.org/
What are some alternatives?
deepnet - Educational deep learning library in plain Numpy.
Rustlings - :crab: Small exercises to get you used to reading and writing Rust code!
tinygrad - You like pytorch? You like micrograd? You love tinygrad! ❤️ [Moved to: https://github.com/tinygrad/tinygrad]
codewars.com - Issue tracker for Codewars
deeplearning-notes - Notes for Deep Learning Specialization Courses led by Andrew Ng.
devops-exercises - Linux, Jenkins, AWS, SRE, Prometheus, Docker, Python, Ansible, Git, Kubernetes, Terraform, OpenStack, SQL, NoSQL, Azure, GCP, DNS, Elastic, Network, Virtualization. DevOps Interview Questions
ML-From-Scratch - Machine Learning From Scratch. Bare bones NumPy implementations of machine learning models and algorithms with a focus on accessibility. Aims to cover everything from linear regression to deep learning.
Scala Exercises - The easy way to learn Scala.
NNfSiX - Neural Networks from Scratch in various programming languages
Demos and Examples in Scala (Chinese) - scala、spark使用过程中,各种测试用例以及相关资料整理
yolov7 - Implementation of paper - YOLOv7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectors
interviews - Everything you need to know to get the job.