neural_network_chess VS alpha-zero-boosted

Basically, MCTS is used in combination with reinforcement learning. This ensures that we get better evaluations for non terminal positions in training games. You might want to have a look at my free book @ https://github.com/asdfjkl/neural_network_chess

So consider instead keeping a running score of the position, rather than recalculating it in full every time. One of the main tricks behind Stockfish NNUE, the current top chess engine, is a clever way of not needing to fully recalculate the full neural net every time, but to do a delta update for each move for a substantial part of the network. I forget the main paper about that, but you can read about it in https://github.com/asdfjkl/neural_network_chess.

My book https://github.com/asdfjkl/neural_network_chess covers some of these topics...

Neural Networks For Chess

Shameless self-plug: I wrote a book about neural networks in chess and you can get a free PDF here https://github.com/asdfjkl/neural_network_chess (no ads, no fremium)

> I came up with a nifty implementation in Python that outperforms the naive impl by 30x, allowing a pure python MCTS/NN interop implementation. See https://www.moderndescartes.com/essays/deep_dive_mcts/

Great post!

Chasing pointers in the MCTS tree is definitely a slow approach. Although typically there are < 900 "considerations" per move for alphazero. I've found getting value/policy predictions from a neural network (or GBDT[1]) for the node expansions during those considerations is at least an order of magnitude slower than the MCTS tree-hopping logic.

[1] https://github.com/cgreer/alpha-zero-boosted

What you can do is checkout the algorithm at a particular stages of development. AlphaZero&Friends start out not being very good at the game, then over time they learn and become super human. You typically checkpoint the weights for the model at various stages. So early on, the algo would be like a 600 elo player for chess and then eventually get to superhuman elo levels. So if you wanted to train you can gradually play against versions of the algo until you can beat them by loading up the weights at various difficulty stages.

I implemented AlphaZero (but not Mu yet) using GBDTs instead of NNs here if you're curious about how it would work: https://github.com/cgreer/alpha-zero-boosted. Instead of saving the "weights" for a GBDT, you save the splitpoints for the value/policy models, but the concept is the same.