osgameclones VS promptbench

On a similar note, released games with open sourced code: https://osgameclones.com/?tag=official

https://torque3d.org was used on Airship Dragoon, Marble Blast Gold, Penny Arcade Adventures, Legions: Overdrive and descends from the original Tribes engine.

Hellbreaker, Bit Champs with https://u3d.io

And plenty of successful games with Ogre, including Torchlight: https://www.ogre3d.org/showcase

If you’re looking for even more

https://osgameclones.com/

There are already a few projects like this (i like switching software), maybe it's better to consolidate the efforts? (or at least provide i link to them like osgameclones).

There's this list of open source reverse engineered games for pc and some console games https://osgameclones.com/

Might want to add it to open source game clones.

Fair question.

Evaluate refers to the phase after training to check if the training is good.

Usually the flow goes training -> evaluation -> deployment (what you called inference). This project is aimed for evaluation. Evaluation can be slow (might even be slower than training if you're finetuning on a small domain specific subset)!

So there are [quite](https://github.com/microsoft/promptbench) [a](https://github.com/confident-ai/deepeval) [few](https://github.com/openai/evals) [frameworks](https://github.com/EleutherAI/lm-evaluation-harness) working on evaluation, however, all of them are quite slow, because LLM are slow if you don't have infinite money. [This](https://github.com/open-compass/opencompass) one tries to speed up by parallelizing on multiple computers, but none of them takes advantage of the fact that many evaluation queries might be similar and all try to evaluate on all given queries. And that's where this project might come in handy.