stable-baselines3 VS wandb

The latest release (v3.0.0) of Upkie's software brings a functional sim-to-real reinforcement learning pipeline based on Stable Baselines3, with standard sim-to-real tricks. The pipeline trains on the Gymnasium environments distributed in upkie.envs (setup: pip install upkie) and is implemented in the PPO balancer. Here is a policy running on an Upkie:

PettingZoo 1.24.0 is now live! This release includes Python 3.11 support, updated Chess and Hanabi environment versions, and many bugfixes, documentation updates and testing expansions. We are also very excited to announce 3 tutorials using Stable-Baselines3, and a full training script using CleanRL with TensorBoard and WandB.

I am wondering why there is so few algorithms in Stable Baselines 3 (SB3, https://github.com/DLR-RM/stable-baselines3/tree/master)? I was expecting some algorithms like ICM, HIRO, DIAYN, ... Why there is no model-based, skill-chaining, hierarchical-RL, ... algorithms implemented there?

Discord is more focused, and they have a page for people who wants to contribute https://github.com/DLR-RM/stable-baselines3/blob/master/CONTRIBUTING.md

Therefore, I debugged this error to the ReplayBuffer that was imported from `SB3`. This is the problem function -

Have you ever wanted to use dm-control with stable-baselines3? Within Reinforcement learning (RL), a number of APIs are used to implement environments, with limited ability to convert between them. This makes training agents across different APIs highly difficult, and has resulted in a fractured ecosystem.

Changelog: https://github.com/DLR-RM/stable-baselines3/releases/tag/v1.8.0

Great project! One question though, is there any reason why you are not using existing RL models instead of creating your own, such as stable baselines?

Weights & Biases — The developer-first MLOps platform. Build better models faster with experiment tracking, dataset versioning, and model management. Free tier for personal projects only, with 100 GB of storage included.

HuggingFace and wandb.ai (both competitors of OpenAI) both also have "do own research"

Wandb is the best! https://wandb.ai/

To track stuff I've used wandb.ai in a company in the past, as someone else pointed out. Regarding metrics... This is really specific to your domain, and it is such a broad question. You could count color pixels, the distribution of intensity histograms, etc etc.

Step 1: The colab notebook uses wandb.ai, so you need to register for a wandb.ai account beforehand if you want to use the colab notebook. After registering you need to go to your homepage and copy the API key and paste/keep it somewhere.

I want to take a look at tools like https://wandb.ai/ and they would integrate into some of the pipelines I'm playing with.

does current cgi and ai tech can bring back leslie nielsen? might use unreal engine and https://www.resemble.ai/ or https://wandb.ai/?

I'm currently working for a MLOps company so I'm heavily using their tools (Weights & Biases) but I've used custom C++ for deployment, Pytorch + fastai for quick experimentation, Weights & Biases for experiment tracking, hyper-parameter tuning + model versioning (hence why I went to work for them), custom database + data pipeline, HoloViz for data visualisation (really nice dashboarding tool), Jenkins for CI/CD, I also love Github Actions.