agents VS Gymnasium

im trying to follow this tutorial = https://github.com/tensorflow/agents/blob/master/docs/tutorials/9_c51_tutorial.ipynb

I followed the TensorFlow tutorial for agents and the multi armed bandit tutorial and now I'm trying to make one of the already implemented agents, from the examples, work on my own environment. Basically my environment exists of 5 actions and 5 observations. Applying one action i results in the same state i. One action contains another step of sending that action number to a different program via a socket and the answer from the program is interpreted for the reward. My environment seems to be working, I used the little test script below to test the observe and action functions. I know this is not a full proof but showed its atleast working.

Depending on what task you pick the "best" algo will vary. There are also a bunch of variations and tricks for each of those, some of which have been given new names over time. If you are working on a project I would suggest whichever one has the simplest and most extendable implementation. If you really want to compare all of them you can use libraries that have them all implemented, such as tfagents.

I'm using a simple training loop, based on this.

Hello everyone, I published my python project a month ago, it's a command line interface for training, tuning and reusing reinforcement learning algorithms in tensorflow 2.x. It's similar to stable-baselines, tf-agents, and not so many others. It seems like it's not getting enough attention despite the README, license, and everything else.

Well it seems it doesn't flatten anything, just passes OrderedDict as input dense. Not sure but apparently it's keras that makes that a list of tensors. You can dig around places like https://github.com/tensorflow/agents/blob/v0.8.0/tf_agents/networks/network.py https://github.com/tensorflow/agents/blob/v0.8.0/tf_agents/agents/dqn/dqn_agent.py https://github.com/openai/gym/blob/master/gym/spaces/dict.py if you want to be really sure.

2- You could go the reinforcement learning approach by controlling these parameters using an agent. This would mean that the parameters would have to change on the fly, which I am not sure if appropriate. If so, creating a gym environment is not so hard, which would then use something like tf.agents , rlax or any other rl framework of your liking.

Not the same style and looks somehow more complicated but i want to mention tf.agents if you don't know about it already.

"Show HN: Ghidra Plays Mario" (2023) https://news.ycombinator.com/item?id=37475761 :

[RL, MuZero reduxxxx ]

> Farama-Foundation/Gymnasium is a fork of OpenAI/gym and it has support for additional Environments like MuJoCo: https://github.com/Farama-Foundation/Gymnasium#environments

> Farama-Foundatiom/MO-Gymnasiun: "Multi-objective Gymnasium environments for reinforcement learning": https://github.com/Farama-Foundation/MO-Gymnasium

https://github.com/Farama-Foundation/Gymnasium#environments

Farama-Foundatiom/MO-Gymnasiun:

https://github.com/Farama-Foundation/Gymnasium - A framework Python library to build and train your own AI to play games

The only change in the environment from the one here is the reward function which is given its return value using the following code snippet (replacing lines 648-672 in the above url):

This may be useful: https://github.com/Farama-Foundation/Gymnasium

Here are all the environments. So for example, if instead of Hopper-v2 you want the acrobat environment from classic control you can write: env = gym.make('Acrobot-v1')

This release also includes a large number of documentation updates, minor bug fixes, and other minor improvements; the full release notes are available here if you’d like to learn more: https://github.com/Farama-Foundation/Gymnasium/releases/tag/v0.28.0.

Thanks! It really depends on the task that you want to implement. But in general, sticking to the standard gymnasium API is important. If you want to implement a 2D environment then PyGame is promising. If it's more like a game, check out Unity ML-Agents or Godot RL Agents. Anything simpler can also be just pure python code. You also need to carefully design your observation space, action space and reward function. My advice is to explore design choices of related environments.

You can read the release notes here: https://github.com/Farama-Foundation/Gymnasium/releases/tag/v0.27.0. You can upgrade from 0.26 without any changes unless you're doing something very uncommon; this is how releases will generally be going forward.