alpha-zero-boosted
mctx
alpha-zero-boosted | mctx | |
---|---|---|
2 | 10 | |
79 | 2,203 | |
- | 1.1% | |
3.2 | 0.0 | |
almost 4 years ago | 3 months ago | |
Python | Python | |
- | Apache License 2.0 |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
alpha-zero-boosted
-
DeepMind has open-sourced the heart of AlphaGo and AlphaZero
> I came up with a nifty implementation in Python that outperforms the naive impl by 30x, allowing a pure python MCTS/NN interop implementation. See https://www.moderndescartes.com/essays/deep_dive_mcts/
Great post!
Chasing pointers in the MCTS tree is definitely a slow approach. Although typically there are < 900 "considerations" per move for alphazero. I've found getting value/policy predictions from a neural network (or GBDT[1]) for the node expansions during those considerations is at least an order of magnitude slower than the MCTS tree-hopping logic.
[1] https://github.com/cgreer/alpha-zero-boosted
-
MuZero: Mastering Go, chess, shogi and Atari without rules
What you can do is checkout the algorithm at a particular stages of development. AlphaZero&Friends start out not being very good at the game, then over time they learn and become super human. You typically checkpoint the weights for the model at various stages. So early on, the algo would be like a 600 elo player for chess and then eventually get to superhuman elo levels. So if you wanted to train you can gradually play against versions of the algo until you can beat them by loading up the weights at various difficulty stages.
I implemented AlphaZero (but not Mu yet) using GBDTs instead of NNs here if you're curious about how it would work: https://github.com/cgreer/alpha-zero-boosted. Instead of saving the "weights" for a GBDT, you save the splitpoints for the value/policy models, but the concept is the same.
mctx
- About Monte Carlo tree search in Jax
-
Programming language dilemma
Maybe you can have your cake and eat it too. :) You could use Python with one of the hardware accelerating languages like Jax. This project for example uses Jax to implement Monte Carlo Tree Search and includes a few games as examples. https://github.com/deepmind/mctx
-
Is there any proof that AlphaZero actually exist?
recently tree search part of alpha zero has gone open source https://github.com/deepmind/mctx
-
[D] Anyone interested in training an AI for Tigris and Euphrates?
You could try starting with https://github.com/deepmind/mctx. You’ll probably need to expose your game state and actions via IPC of some sort or FFI your rust code to Python.
-
DeepMind has open-sourced the heart of AlphaGo and AlphaZero
Interesting approach to private variables https://github.com/deepmind/mctx/blob/577fc77a3cda1b796e277e...
- AlphaZero's Monte Carlo tree search implementation in Jax
-
Anyone found any working replication repo for MuZero?
Just have a look at the DM repo: https://github.com/deepmind/mctx
- MuZero Implementation
- Official DeepMind MuZero Implementation
-
Finally an official MuZero implementation
deepmind/mctx: Monte Carlo tree search in JAX (github.com)
What are some alternatives?
KataGo - GTP engine and self-play learning in Go
EfficientZero - Fork of EfficientZero to use newer libraries and to fix a few runtime bugs. Also includes pretrained models!
neural_network_chess - Free Book about Deep-Learning approaches for Chess (like AlphaZero, Leela Chess Zero and Stockfish NNUE)
minihack - MiniHack the Planet: A Sandbox for Open-Ended Reinforcement Learning Research
katrain - Improve your Baduk skills by training with KataGo!
adversarial-robustness-toolbox - Adversarial Robustness Toolbox (ART) - Python Library for Machine Learning Security - Evasion, Poisoning, Extraction, Inference - Red and Blue Teams
leela-zero - Go engine with no human-provided knowledge, modeled after the AlphaGo Zero paper.
craftingway - A ffxiv crafting tool
mars - Mars is a tensor-based unified framework for large-scale data computation which scales numpy, pandas, scikit-learn and Python functions.
omega - A number of agents (PPO, MuZero) with a Perceiver-based NN architecture that can be trained to achieve goals in nethack/minihack environments.