bet VS returnn-experiments

Indeed! In fact, I have a project [0] from last year that uses a GPT-style transformer to address that exact issue :) However, it’s hard to go far outside simulations in real home robotics without a good platform, out of which efforts came Dobb-E.

[0] https://mahis.life/bet/

Hi HN!

First author of the paper here, thought some of you may enjoy reading about this! Even now, training robots on human demonstration data is the best way to get them to do new and exciting things in the real world. However, this generally requires a lot of data curation in the standard way: the robots can only follow along if you give them data that is solving a single task in a single way.

To improve the status quo, we introduce Behavior Transformer in this paper, which can learn from unlabeled demonstration data solving multiple different tasks in different ways using a GPT-like generator model. We had to make some modifications to fit the continuous actions, unlike the standard GPT model which fits discrete words.

As it turns out, unconditional rollouts from this model shows a lot more "natural" behavior (i.e. different tasks solved in different rollouts in different ways)_than standard behavioral cloning. More importantly, behavior transformers show much better mode coverage compared to previous models, and show some level of compositionality. Check out our videos! [1]

Finally, another oft-ignored part I am quite proud of is our code release -- we worked quite hard to make sure our code [2] is easy to read, reproduce, and remix! And also, did I tell you that these models train super fast? The Franka Kitchen environment in the top video [3] takes just 10 minutes on an Nvidia 3080 to the point you are seeing in the video. Compare that with standard RL training, and you might agree with me that a small number of demonstrations can truly go a long way!

Happy to answer questions, as well! Have a great Friday, wherever you are :)

[1] https://mahis.life/bet

[2] https://github.com/notmahi/bet

[3] https://mahis.life/bet/more/kitchen/

The code is all released already. You find it here: https://github.com/rwth-i6/returnn-experiments/tree/master/2...

This is TensorFlow-based. But I also have another PyTorch-based implementation already, also public (inside our other repo, i6_experiments). It's not so easy currently to set this up, but I'm working on a simpler pipeline in PyTorch.

We don't have the models online yet, but we can upload them later. But I'm not sure how useful they are outside of research, as they are specifically for those research tasks (Librispeech, Tedlium), and probably don't perform too well on other data.

This works for an architecture which has been well tuned and studied before, like LSTM or Transformer.

Once you do research on the model, testing out things, it often tends to become such kwarg monster in many frameworks.

Having everything (relevant) in one file (even in the config file itself with hyper params) allows you to copy the file for every experiment and modify it inplace. This avoids the kwargs mess. But then the config files are very complex, and can become messy in other ways (esp for research projects). Example: https://github.com/rwth-i6/returnn-experiments/blob/master/2...

Such approach makes it much more flexible and does not mess with the baseline code. As you say, it's more like an evolutionary DNA-like approach, where you then tend to do crossovers with other evolved good-performing configs, etc.