action-branching-agents
pytorch-a2c-ppo-acktr-gail
action-branching-agents | pytorch-a2c-ppo-acktr-gail | |
---|---|---|
2 | 3 | |
105 | 3,423 | |
- | - | |
0.0 | 0.0 | |
over 1 year ago | almost 2 years ago | |
Python | Python | |
MIT License | MIT License |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
action-branching-agents
-
Large Action Spaces
Exactly, multiple action heads. There are some works that try this for DQN as https://arxiv.org/abs/1711.08946. However, i have not tried since I tend to prefer actor-critic methods.
-
(Newbie question)How to solve using reinforcement learning 2x2 rubik's cube which has 2^336 states without ValueError?
That's a larger number than the number of atoms in the Universe. You need some kind of branching to limit the actions. Check out: https://github.com/atavakol/action-branching-agents
pytorch-a2c-ppo-acktr-gail
-
How does advantage estimation is done when episodes are of variable length in PPO?
As an example look at "compute_returns" function here (and pay attention to how self.masks is used): https://github.com/ikostrikov/pytorch-a2c-ppo-acktr-gail/blob/master/a2c_ppo_acktr/storage.py
-
How to pretrain a model on expert data?
Try using an imitation learning algorithm. Two popular options are MaxEnt IRL and GAIL. This repository has GAIL implementation and this repository has MaxEnt IRL and GAIL implementation. There are other implementations too that you can check out.
-
Trying to Train PPO Agent for Pendulum-v0 from Pixel Inputs
For the PPO, I used this repo, which includes most tricks including GAE, normalized rewards, etc. I have verified this repo works for the traditional Pendulum-v0 task and Atari games (Pong and Breakout).
What are some alternatives?
ai-traineree - PyTorch agents and tools for (Deep) Reinforcement Learning
Super-mario-bros-PPO-pytorch - Proximal Policy Optimization (PPO) algorithm for Super Mario Bros
tensorforce - Tensorforce: a TensorFlow library for applied reinforcement learning
soft-actor-critic - Implementation of the Soft Actor Critic algorithm using Pytorch.
trax - Trax — Deep Learning with Clear Code and Speed
dm_control - Google DeepMind's software stack for physics-based simulation and Reinforcement Learning environments, using MuJoCo.
acme - A library of reinforcement learning components and agents
Deep-Reinforcement-Learning-in-Large-Discrete-Action-Spaces - PyTorch implementation of the paper "Deep Reinforcement Learning in Large Discrete Action Spaces" (Gabriel Dulac-Arnold, Richard Evans, Hado van Hasselt, Peter Sunehag, Timothy Lillicrap, Jonathan Hunt, Timothy Mann, Theophane Weber, Thomas Degris, Ben Coppin).
TensorFlow2.0-for-Deep-Reinforcement-Learning - TensorFlow 2.0 for Deep Reinforcement Learning. :octopus:
PCGrad - Code for "Gradient Surgery for Multi-Task Learning"
DI-engine - OpenDILab Decision AI Engine