[D] Adding a new RL environment to envpool

• envpool

  3 1,017 4.2 C++

  C++-based high-performance parallel environment execution engine (vectorized env) for general RL environments.

  

Envpool provides high parallelization of RL environments. Unfortunately, there are still many environments that are not supported by them. One of them is FrankaKitchen of D4RL, a library for offline RL.

• D4RL

  2 1,193 0.0 Python

  A collection of reference environments for offline reinforcement learning

  

Envpool provides high parallelization of RL environments. Unfortunately, there are still many environments that are not supported by them. One of them is FrankaKitchen of D4RL, a library for offline RL.

• InfluxDB

  www.influxdata.com sponsored

  Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

  

• open_spiel

  44 3,999 9.5 C++

  OpenSpiel is a collection of environments and algorithms for research in general reinforcement learning and search/planning in games.

  

I looked at the paper before and thought some of the benchmarks in the paper were cherry picked and that the whole let’s write env in compiled languages things has been done a lot already and only works well for games / physics / environments that are extremely well defined . I usually end up back in ray env bc I usually need features like parametric action spaces, or predictive models built inside environments, or historical data. Codingwise I looked at the example file that they had and felt like it wasn’t for me bc I don’t like Bazel ( I use pants for python monorepo ) and the c++ api was overly verbose for me. I don’t feel like I could implement reward shaping or env business logic in a very clear way. I don’t do much c++ dev work though and someone who is a seasoned c++ dev might not care. I liked the openspiel c++ api for env a lot more. https://github.com/deepmind/open_spiel/blob/master/docs/developer_guide.md I have found that building a new env can sometimes be frustrating to debug and I would probably debug a python env before converting to an openspiel/envpool env if I had to use a C++ env for a new problem.

Suggest a related project