berserk VS fishtest

Yep, completely normal. You should now do two things. First try another engine like Ethereal as suggested by IMJorose, Berserk or any other engine; and second, try simulating Chessbase-Stockfish communication by hand.

it's all cool but at stockfish 3 times fishtest racked up insane 30 cores or so :) Overall it growth in strength attracted more people who have more CPUs and it made it stronger which attracted more people etc. At times I started to contribute to sf mibere was considered a god of hardware with 200~ cores running, but now we have noob, technologov who ALWAYS run 4x of this and also mlang who powers up 1000 cores when he feels like it + people like linrock with 100 cores, etc. Sure, it made an impact, but what I'm saying that even at sf 3 times this number of cores was enough to develop it really rapidly. As Tord (author of Glaurung) said making top engine being open-source opened floodgates in terms of progress, because when he started to be intereted in chess programming progress was pretty miserable since everything was commercial and ideas exchange were basically non-existent. Nowadays field is all about open-source, everyone feels free to try other people ideas. And stockfish is like a leading flashlight because truth to be told almost 0 ideas from other open source engines work in it while vice versa it's more of a case. I even talked to other engine devs which resulted in some of my contribution in other open-source engines, in classical eval it was double pawn protection of king ring which spread among I think like 5 engines, now in search it's https://github.com/jhonnold/berserk/blob/main/src/search.c#L565 Also razoring in berserk is based on my form that was good in sf but didn't quite make it, but there it was https://github.com/jhonnold/berserk/commit/3a55e560e0b45900802e424a9c29aecbfd371fe5 truth to be told I have absolutely 0 idea why it was good in Berserk but not good enough in Stockfish but oh well :)

you can have a look at openbench maybe, https://github.com/AndyGrant/OpenBench . It is currently a testing framework for chess engines (only invited ones) or fishtest. https://tests.stockfishchess.org/tests Stockfish testing framework. Both of these test if change A beats the master version using statistics. Both look at the git repository and download it and create/compile the engine, that might be another solution for you.

[4]: https://github.com/glinscott/fishtest/wiki/Regression-Tests#...

FishTest has thousands of computers running 24/7 https://tests.stockfishchess.org/tests. Nerds at gcp,aws,universities burn thousands of GPU hours on lc0-client just to watch their GPUs warm up. They aren't just attaching 20tb of endgames to an old macbook.

if removing any rng from stockfish would gain elo it would be merged. If you think it gains elo feel free to submit a test to https://tests.stockfishchess.org/tests , but it doesn't so what your saying makes no sense.

Where did you get these numbers from? https://github.com/glinscott/fishtest/wiki/Regression-Tests shows an elo difference of 351.3 elo points. This is further exaggerated by the fact that sf9 lacks nnue and is particularly weak at the starting position and consequently 960 starting positions.

In https://github.com/glinscott/fishtest/wiki/Regression-Tests you can see 8moves_v3.pgn opening book (09/11/2013 - current) with a link:

You can see the various code changes in https://github.com/glinscott/fishtest/wiki/Regression-Tests . Both the actual change and the elo change from the code change.

Current progression test is still in progress but you can see all of the previous ones here: https://github.com/glinscott/fishtest/wiki/Regression-Tests