AgileRL
hlb-gpt
AgileRL | hlb-gpt | |
---|---|---|
12 | 5 | |
501 | 253 | |
4.2% | - | |
9.8 | 3.7 | |
5 days ago | about 2 months ago | |
Python | Python | |
Apache License 2.0 | Apache License 2.0 |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
AgileRL
- [P] Introducing PPO and Rainbow DQN to our super fast evolutionary HPO reinforcement learning framework
- Introducing PPO and Rainbow DQN to our super fast evolutionary HPO reinforcement learning framework
-
[P] Significant improvements for multi-agent reinforcement learning!
Please check it out! https://github.com/AgileRL/AgileRL
- 10x faster reinforcement learning hyperparameter optimization than SOTA - now with distributed training!
- [P] 10x faster reinforcement learning hyperparameter optimization than SOTA - now with distributed training!
-
(1/2) May 2023
Deep Reinforcement Learning library focused on improving development by introducing RLOps - MLOps for reinforcement learning (https://github.com/AgileRL/AgileRL)
-
[P] 10x faster reinforcement learning HPO - now for RLHF!
https://github.com/AgileRL/AgileRL/blob/main/CONTRIBUTING.md Has a link to our discord too
- 10x faster reinforcement learning HPO - now with CNNs!
- [P] 10x faster reinforcement learning HPO - now with CNNs!
-
[P] Reinforcement learning evolutionary hyperparameter optimization - 10x speed up
GitHub: https://github.com/AgileRL/AgileRL
hlb-gpt
-
In Defense of Pure 16-Bit Floating-Point Neural Networks
As a practitioner specializing in extremely fast-training neural networks, seeing a paper in 2023 considering fp32 as a gold standard over pure non-mixed fp16/bp16 is a bit shocking to me and feels dated/distracting from the discussion. They make good points but unless I am hopelessly misinformed, it's been pretty well established at this point in a number of circles that fp32 is overkill for the majority of uses for many modern-day practitioners. Loads of networks train directly in bfloat16 as the standard -- a lot of the modern LLMs among them. Mixed precision is very much no longer needed, not even with fp16 if you're willing to tolerate some range hacks. If you don't want the range hacks, just use bfloat16 directly. The complexity is not worth it, adds not much at all, and the dynamic loss scaler a lot of people use is just begging for more issues.
Both of the main repos that I've published in terms of speed benchmarks train directly in pure fp16 and bf16 respectively without any fp32 frippery, if you want to see an example of both paradigms successfully feel free to take a look (I'll note that bf16 is simpler on the whole for a few reasons, generally seamless): https://github.com/tysam-code/hlb-CIFAR10 [for fp16] and https://github.com/tysam-code/hlb-gpt [for bf16]
Personally from my experience, I think fp16/bf16 is honestly a bit too expressive for what we need, fp8 seems to do just fine and I think will be quite alright with some accommodations, just as with pure fp16. The what and the how of that is a story for a different day (and at this point, the max pooling operation is basically one of the slowest now).
You'll have to excuse my frustration a bit, it just is a bit jarring to see a streetsign from way in the past fly forward in the wind to hit you in the face before tumbling on its merry way. And additionally in the comment section the general discussion doesn't seem to talk about what seems to be a pretty clearly-established consensus in certain research circles. It's not really too much of a debate anymore, it works and we're off to bigger and better problems that I think we should talk about. I guess in one sense it does justify the paper's utility, but also a bit frustrating because it normalizes the conversation as a few notches back from where I personally feel that it actually is at the moment.
We've got to move out of the past, this fp32 business to me personally is like writing a Relu-activated VGG network in Keras on Tensorflow. Phew.
And while we're at it, if I shall throw my frumpy-grumpy hat right back into the ring, this is an information-theoretic problem! Not enough discussion of Shannon and co. Let's please fix that too. See my other rants for x-references to that, should you be so-inclined to punish yourself in that manner.
-
Neural Networks: Zero to Hero
I made a smaller GPT model that started from Andrej's code that converges to a decent loss in a short amount of time on an A100 -- just under 2.5 minutes or so: https://github.com/tysam-code/hlb-gpt
With the original hyperparameters, it was 30-60 minutes, with a pruned down network and adjusted hyperparameters, about 6 minutes, and a variety of optimizations beyond that to bring it down.
If you want the nano-GPT basically feature-identical (but pruned down) version, 0.0.0 at ~6 minutes or so is your best bet.
You can get A100s cheaply and securely through Colab or LambdaLabs.
-
[P] 10x faster reinforcement learning HPO - now with CNNs!
Check it out! If LLMs are your thing, I did basically the same thing, but for 3.8 val loss on WikiText-103 in maybe 2.3ish minutes or so on an A100: https://github.com/tysam-code/hlb-gpt.
-
MIT 6.S191: Recurrent Neural Networks, Transformers, and Attention
https://github.com/tysam-code/hlb-gpt
Both of these implementations are pretty straightforward for what they do but CIFAR-10 has less dynamic scheduling and stuff so it might be easier to fit in your head. However, both are meant to be simple (and extremely hackable if you want to poke around and take apart some pieces/add different watchpoints to see how different pieces evolve, etc. I am partially inspired by, among many things, one of those see-through engine kits that I saw in a magazine growing up as a child that I thought was a very cool, dynamic, and hands-on way to just watch how the pieces moved in a difficult topic. Sometimes that is the best way that our brains can learn, though we are all different and learn best differently through different mediums in my experience).
Feel free to let me know if you have any specific questions and I'll endeavor to do my best to help you here. Welcome to an interest in the field!
I guess to briefly touch on one topic -- some people focus on the technical only first, like backprop, and though math is required heavily for more advanced research, I don't learn concepts very well through details only. Knowing that backprop is "Calculate the slope for the error in this high-dimensional space for how a neural network was wrong at a certain point, then take a tiny step towards minimizing the error. After N steps, we converge to a representation that is like a zip file of our input data within a mathematical function" is probably enough for 90-95% of the usecases you will do as a ML practitioner, if you do so. The math is cool but there are more important things to sweat over IMO, and I think messaging to the contrary raises the barrier to entry to the field and distracts from the important things, which we do not need as much. It's good to learn after you have space in your brain for it after you understand how the whole thing works together, though that is just my personal opinion after all.
Much love and care and all that and again feel free to let me know if you have any questions please. :) <3
-
[P] Introducing hlb-gpt: A rapid prototyping toolbench in <350 lines of code to speed up your LLM research exploration
You can find the code for hlb-gpt here: https://github.com/tysam-code/hlb-gpt
What are some alternatives?
chat-ui - Open source codebase powering the HuggingChat app
hlb-CIFAR10 - Train CIFAR-10 in <7 seconds on an A100, the current world record.
RLeXplore - RLeXplore provides stable baselines of exploration methods in reinforcement learning, such as intrinsic curiosity module (ICM), random network distillation (RND) and rewarding impact-driven exploration (RIDE).
micrograd - A tiny scalar-valued autograd engine and a neural net library on top of it with PyTorch-like API
loopquest - A Production Tool for Embodied AI
randomfun - Notebooks and various random fun
de-torch - Minimal PyTorch Library for Differential Evolution
makemore - An autoregressive character-level language model for making more things
Muzero - Pytorch Implementation of MuZero for gym environment. It support any Discrete , Box and Box2D configuration for the action space and observation space.
nanoGPT - The simplest, fastest repository for training/finetuning medium-sized GPTs.
q-learning-algorithms - This repository will aim to provide implementations of q-learning algorithms (DQN, Double-DQN, ...) using Pytorch.
Open-Llama - The complete training code of the open-source high-performance Llama model, including the full process from pre-training to RLHF.