SuperSuit VS stable-baselines

"SuperSuit is a library that includes all commonly used wrappers in RL (frame stacking, observation, normalization, etc.) for PettingZoo and Gym environments with a nice API. We developed it in lieu of wrappers built into PettingZoo. https://github.com/Farama-Foundation/SuperSuit "

+1 on PettingZoo, and the wrappers they provide as SuperSuit come in handy as well!. Also check out OpenSpiel

Multi-agent isn’t supported by default in stable baselines, but you can make it work with PettingZoo. This example trains a single policy to control every agent in an environment (Parameter sharing). You could use these SuperSuit wrappers to work with other methods (self-play, independent learning, etc) but you would probably need to write some custom training code. https://github.com/PettingZoo-Team/SuperSuit#parallel-environment-vectorization

As underlined by gold-panda, you can give a try with multiprocessing. I once implemented a version based on what is done in stable_baselines v1 (https://github.com/hill-a/stable-baselines/blob/master/stable_baselines/common/vec_env/subproc_vec_env.py)

Found relevant code at https://github.com/hill-a/stable-baselines + all code implementations here

Depends what you wanna do. Universal answer would be https://stable-baselines.readthedocs.io/

As you will see here it is recommended to augment this safety measure with target kl_divergence, that will ensure even smoother learning and enforce early stopping to prevent learning collapses.

Code for https://arxiv.org/abs/1707.06347 found: https://github.com/hill-a/stable-baselines

I haven't read the code, but stable-baselines doesn't support multi-agent environments (https://github.com/hill-a/stable-baselines/issues/423), so I think they're trying to make learning multi-agent easier with Environment.train().

- https://github.com/hill-a/stable-baselines (Tensorflow)

- tf2 speed: https://github.com/hill-a/stable-baselines/issues/576#issuecomment-573331715