o3de VS GodotAIGym

It's odd to me that when the whole Unity fiasco happened, everyone was basically looking at either Godot or Unreal, but pretty much nobody mentioned or cared for something like O3DE.

If you praise Godot for being open source a lot, then it stands to reason that you should similarly prefer O3DE as opposed to Unreal: https://github.com/o3de/o3de/blob/development/LICENSE.txt (no idea why they're going for both Apache 2 and MIT license, though) vs https://www.unrealengine.com/en-US/license

Unless people just care about the options that are popular enough to warrant their attention and the features that they provide, whereas the licensing is actually a boon, rather than the main factor, given that Unreal also did some slight price increases a while later as well: https://www.unreal-university.blog/post/unreal-engine-5-pric...

Either way, it's still nice to have lots of options available regardless of the licensing details (though this kind of does fragment developers among bunches of different projects), be it Godot, O3DE, Stride, Unreal or even something like jMonkeyEngine (one of the rare Java engines/editors with 3D) or NeoAxis (that one had a cool voxel LOD solution, but performance on AMD hardware was bad).

03DE: Open source game engine, under Apache License 2.0, developed by Amazon and the linux foundation. Seems to work under a modular package called "gems", that you can use to pull in the functionality you need. It uses c++ as it's main language, but you can use Lua, python or visual scripting for scripting stuff. Has multiplayer built into the engine and what they call a "robust" system for open-world games. There seems to be a lot of tutorials on the site, but they aren't laid out great.

Hello, there are already Unreal and Unity engines bindings for python. I just made my own for Godot engine: https://github.com/lupoglaz/GodotAIGym And also recorded tutorials on how to code LunarLander environment and train an agent:

If you're a fan of deep reinforcement learning and Godot engine, I made a tutorial series on how to train your AI in Godot using GodotAIGym (https://github.com/lupoglaz/GodotAIGym) library:

Update GodotAIGym for windows and make youtube tutorial similar to HeartBeast action rpg tutorial on how to code bipedal walker (like this) environment and train an agent from scratch. In general I think that gamedev misses a lot of interesting recent AI techniques so, I want to make tutorials on communicating agents, learning locomotion in 3d etc.

There’s also an interface to use OpenAI Gym environments in Godot.

https://github.com/lupoglaz/GodotAIGym

I have succeeded in modifying the new interface to send images to a version of Google’s DreamerV2 RL learner. I have a simple test environment I could publish.

But I stalled because I kept getting a segmentation fault and I wasn’t up for debugging it.

I think it would be very cool to be able to make a game in Godot and then train an agent to play it using DreamerV2 or similar. I’ve still not posted my sample files but if someone responds here saying they would like to see it, I will try to push it all to GitHub.

Great comment thank you.

Yes unless the industry sees value in a step change in the scale on offer to regular devs, progress on massive nets will be slow.

Hooking things together is pretty much my job. I have used ROS extensively in the past but now I just hook things together using python.

But I consider what Tesla is doing to be pretty promising, and they are layering neural nets together where the output of three special purpose networks feed in to one big one etc. They call that a hydra net. No framework like ROS is required because each net was trained in situ with the other nets on the output of those nets, so I believe all compute logic is handled within the neural network processor (at some point they integrate standard logic too but a lot happens before that). Definitely watch some Karpathy talks on that.

And currently I am simply not skilled enough to compose multiple networks like that. So I COULD compose multiple standalone networks, process them separately, and link them together using IPC of some kind, but it would be very slow compared to what's possible.

I probably will do that eventually. I mean right now I am working on two networks - an RL trained trail following network trained in simulation on segmentation-like data (perhaps using Dreamer V2), and a semantic segmentation net that is trained on my hand labeled dataset with "trail/not-trail" segmentation. So far my segmentation net works okay. And a first step will actually be to hand-write an algorithm to go from segmentation data to steering. My simulation stuff is almost working. I built up a training environment using Godot video game engine and hacked the shared memory neural net training add on to accept image data, but when I run the sim in training on DreamerV2, something in the shared memory interface crashes and I have not resolved it. [1]

But all of this is a hobby and I have a huge work project [2] I am managing myself that is important to me, so the self driving off road stuff has been on pause. But I don't stress about it too much because the longer I wait, the better my options get on the neural network side. Currently my off road rover is getting some mechanical repairs, but I do want to bring it back up soon.

[1] https://github.com/lupoglaz/GodotAIGym/issues/15

[2] https://community.twistedfields.com/t/a-closer-look-at-acorn...

Godot Open AI Gym Training (Linux Only)