rl-baselines3-zoo VS pybullet-gym

I'm trying to solve MountainCar-v0 enviroment from gymnasium with the A2C algorithm and the agent doesn't find a solution. I checked this so I added import stable_baselines3.common.sb2_compat.rmsprop_tf_like as RMSpropTFLike. Also checked the rl-baselines3-zoo for the hyperparameter tuning. So my code is:

RL Zoo3 (training framework): https://github.com/DLR-RM/rl-baselines3-zoo

I'm still new to the domain but wanted to shared some experimental data I've gathered from massive amount of experimentation. I don't have a strong understanding of the theory as I'm more of a software engineer than data scientist, but perhaps this will help other implementers. These notes are based on Stable Baselines 3 and RL Baselines3 Zoo with using PPO+LSTM (should apply generally to all the algos for the most part)

Try hyperparameter search. It's implemented here: https://github.com/DLR-RM/rl-baselines3-zoo for stable-baselines3. Hyperparameters make a huge difference in RL, much more than in supervised learning.

Integrating RL-baselines3-zoo

Hello, I've been running some tests of Double DQN with Stable Baselines 3 Zoo and to compare I'm using the graphs provided by Noisy Networks For Exploration.

- you can find tuned hyperparameters for DDPG, SAC, PPO in https://github.com/DLR-RM/rl-baselines3-zoo

For more complete implementation: https://github.com/DLR-RM/rl-baselines3-zoo

https://github.com/DLR-RM/rl-baselines3-zoo (PyTorch based, using https://github.com/DLR-RM/stable-baselines3)

We also release 100+ trained models in our experimental framework, the rl zoo: https://github.com/DLR-RM/rl-baselines3-zoo

And when something like "RL/Fortnite Racing" comes out, with shared inventory, it may work as the pilot "beta test" of RL finally working in UE5. At least the game's physics engine (pybullet.org) is independent of the graphics. https://x.com/CactousMan/status/1690437803711320064

From what I know in pybullet you define the system of the environment loading an urdf file. So if you check the InvertedPendulum environment of pybullet I think you could create your desired urdf without a problem. I have not used mujoco, but it seems strange that it could not do what you want to do... Maybe there is a parameter of the joints or some kind of tolerance for the initial position?

So far I found PyBullet, RobotPy, RobotDK, SOFA, and some others, but I wonder if there is something that is comparable or better than Unity 3D for this specific use case.

yeah, i guess compared to modeling tools like tinkercad, blender is not intuitive and hard to learn. im not aware of any FEM with blender but there's bullet, basis of https://pybullet.org/. blender has an omniverse connector now so it could probably be used in a pipeline with nvidia's isaac. https://developer.nvidia.com/isaac-sim https://developer.nvidia.com/blog/introducing-omniverse-usd-support-for-blender/

Programming is also really important when learning anything about machine learning and I would start out with the OpenAI Gym and its derivative for learning some of the simpler algorithms on toy environments that doesn't require a whole lot of computing power. Then you can move on to the more hardware constrained simulators that I mentioned in the previous comment (MuJoCo, PyBullet, NVIDIA Isaac Gym).

shout-out to the open-source clones of the mujoco gym environments here