iris VS motion-diffusion-model

Yea, after all these LLMs are predicting one sequence of tokens from another sequence of tokens and the tokens could be anything, it just "happens" that text has the most knowledge and the easiest to input, then there are image, sound, video, but tokens could also be learned from world experience in RL:

Transformers are Sample-Efficient World Models:

https://github.com/eloialonso/iris#transformers-are-sample-e...

Transformers are Sample Efficient World Models https://github.com/eloialonso/iris https://arxiv.org/abs/2209.00588v1

This is actually a pretty neat, self-contained implementation that can super easily extended beyond stereotypical natural language models, for example to create world models for video games [1] or to create robot models that can learn to imitate from large, chaotic human demonstration data [2] (disclaimer, I'm an author on the second one.) Basically, GPT (or minGPT) models are EXCELLENT sequence modelers, almost to the point where you can throw any sensible sequence data at it and hope to get interesting results, as long as you don't overfit.

Even though I have only been working on machine learning for around six years, it's crazy to see how the landscape has changed so fast so recently, including diffusion models and transformers. It's not too much to say that we might expect more major breakthroughs by the end of this decade, and end in a place we can't even imagine right now!

[1] https://github.com/eloialonso/iris

Text to Motion is fascinating (and new to me). There's a video and github here :- https://youtu.be/rVkIDj5wgjs and github at https://github.com/GuyTevet/motion-diffusion-model

Here you go :D - https://github.com/GuyTevet/motion-diffusion-model

You can find the Github repo for Motion Diffusion here - https://github.com/GuyTevet/motion-diffusion-model where we're actively working with other devs and the original author to push the code further :)

Human Motion Diffusion