lc0 VS fishtest

So according to the Releases page at https://github.com/LeelaChessZero/lc0/releases on the 1st of January 2022, Lc0 was in version 0.28.2. That takes care of the [White ""] and [Black ""] headers. To get the ratings, I went to the CCRL and looked it up: https://ccrl.chessdom.com/ccrl/404/cgi/compare_engines.cgi?family=Leela%20Chess

Lc0 0.29.0 - released - A beautiful gift for Mac users. (chessengeria.com) Release v0.29.0 · LeelaChessZero/lc0 (github.com)

Sorry, I should've said it in a more clear way. The feature isn't broken in general, but it wasn't intended to be used for randomizing move selection by policy value (normally it uses number of visits, if I understand correctly). This was changed in this PR so I don't think there's any issue that needs to be reported.

UCI is designed to interface with GUIs. He needs it to be usable in "batch mode" which means providing input games/files on a command-line. I just looked at the lc0 documentation. There doesn't seem to be any way of integrating a batch mode https://github.com/LeelaChessZero/lc0/blob/master/FLAGS.md

If you are curious where this game is from, this is actually a computer chess game that I ran on my machine between Leela Chess Zero 0.29.0 prerelease on https://github.com/LeelaChessZero/lc0/releases (CUDNN) playing with the black pieces, and Stockfish 22060708 64X BMI2 on https://abrok.eu/stockfish playing with the white pieces. The game continued 17. Bxe6 Kxe6 18. f4 Nxf4 19. Qg4+ Kd5 20. Rxf4 gxf4. If you check that position with Stockfish 15 NNUE on chess.com, it should realize that it is winning.

For example: https://github.com/LeelaChessZero/lc0/wiki/Contributing-Training-Games

I suspect in a PUCTS based engine, simply changing the parameters to make the search wider could have the effect you want. Ie, the engine will try losing moves more often, which reduces the expected value of a node if it doesn't have a lot of similar options. I haven't checked Leela out in a while, but there is a good chance you don't even need to modify code to try that. https://github.com/LeelaChessZero/lc0

you can have a look at openbench maybe, https://github.com/AndyGrant/OpenBench . It is currently a testing framework for chess engines (only invited ones) or fishtest. https://tests.stockfishchess.org/tests Stockfish testing framework. Both of these test if change A beats the master version using statistics. Both look at the git repository and download it and create/compile the engine, that might be another solution for you.

[4]: https://github.com/glinscott/fishtest/wiki/Regression-Tests#...

FishTest has thousands of computers running 24/7 https://tests.stockfishchess.org/tests. Nerds at gcp,aws,universities burn thousands of GPU hours on lc0-client just to watch their GPUs warm up. They aren't just attaching 20tb of endgames to an old macbook.

if removing any rng from stockfish would gain elo it would be merged. If you think it gains elo feel free to submit a test to https://tests.stockfishchess.org/tests , but it doesn't so what your saying makes no sense.

Where did you get these numbers from? https://github.com/glinscott/fishtest/wiki/Regression-Tests shows an elo difference of 351.3 elo points. This is further exaggerated by the fact that sf9 lacks nnue and is particularly weak at the starting position and consequently 960 starting positions.

In https://github.com/glinscott/fishtest/wiki/Regression-Tests you can see 8moves_v3.pgn opening book (09/11/2013 - current) with a link:

You can see the various code changes in https://github.com/glinscott/fishtest/wiki/Regression-Tests . Both the actual change and the elo change from the code change.

Current progression test is still in progress but you can see all of the previous ones here: https://github.com/glinscott/fishtest/wiki/Regression-Tests