ROS-TCP-Endpoint
ros-sharp
ROS-TCP-Endpoint | ros-sharp | |
---|---|---|
2 | 7 | |
161 | 929 | |
1.9% | 1.0% | |
0.0 | 2.6 | |
about 2 months ago | 4 months ago | |
Python | C# | |
Apache License 2.0 | Apache License 2.0 |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
ROS-TCP-Endpoint
-
Mixed reality game using ROS and Unity [Update 1]
Hey everyone! I wanted to share a project I'm working on to make an FPV drone racing game using Unity and ROS. I'm still pretty early in the process, but the goal I'm working towards is a PC game (maybe VR) that let's you build custom virtual racetracks in an indoor environment and then race actual physical drones (and eventually autonomous) on them. Why I'm building this I like working on robotics projects in my spare time, and one project I've wanted to do for a while has been building my own autonomous drone. I've worked on some systems like that in the past and they've go been really cool to see in person. Along the way, I also started getting into flying FPV drones as well and realized that flying them manually is just as fun as seeing them fly themselves, so I wanted to see if I could combine the two in some way by possibly making a game out of it. Btw, definitely check out the work done at University of Zurich if you're interested in [high-speed autonomous drones](https://rpg.ifi.uzh.ch/aggressive_flight.html). How does it work I put together a quick demo video just to document the current state of my prototype. [Update Video](https://youtu.be/zxoPFM5ol7o. I'm very early in the process, and honestly, I've kind of cheated a bunch just to get something up and running and feel out the concept. Most of what I've done has just been connecting pieces together using off-the-shelf hardware/software. Right now, the prototype basically just proves out the concept of rendering the realtime position of a drone inside of a Unity game and getting all the "piping" set up to get data into the right place. Currently, the information flow is all one-directional from the drone to the PC. On the hardware-side, I'm using Bitcraze's crazyflie drone with it's lighthouse positioning deck and steamVR's base stations for estimating the drone's 3D position. State estimation is pretty hard, but thanks to all the hardwork done by the crazyflie open source community, this is just kind of works out of the box and in realtime (i.e. one of the big reasons why it kind of feels like cheating lol). Communication between the crazyflie and the PC is done using the crazyflie radio dongle. On the software-side, I'm using ROS to handle all the intermediate messaging and obviously Unity for the user interface, game logic and visualization. Challenges I've run into so far Getting the state estimate data from the crazyflie into Unity was somewhat interesting to figure out. Basically, the crazyflie computes its 6DoF pose (position and orientation) onboard, then transmits this telemetry over radio to the PC. On the PC, I wrote a simple ROS publisher node that listens for these messages and then publishes them onto a ROS network. To get the data into Unity, I'm using Unity's ROS-TCP-Connector package (and ROS-TCP-Endpoint) which essentially just forwards the messages from the ROS network into Unity. Inside Unity, I wrote a simple script tied to a gameobject representing the drone that takes the data, transforms it into Unity's coordinate frame and uses it to set the gameobject's position. Overall, it's just a lot of forwarding of information (with some annoying coordinate frame transforms along the way). Another important piece of the puzzle (as far as rendering the drone inside a 3D virtual replica of my room) was building the room model and calibrating it to my actual room. I can go into it more detail for sure, but at a high-level I basically just picked a point in my room to be the origin in both the physical and virtual room, put the crazyflie there (aligned with the axes I picked for the origin) used the crazyflie cfclient tool to center the base station position estimates there. My process was pretty rough as a first pass, and it will very likely have to improve, especially as I move in the mixed reality direction and start rendering virtual objects on a live camera feed. What's next? Tactically, the next few steps would be to add the FPV view into the game (streaming video data from the drone and rendering it into Unity), which involves more data forwarding (and calibration). In addition, I need to add input controls so you can actually fly the drone. The bigger goals in store would be around building out proper gameplay, integrating in autonomy (and figuring out where it makes sense), and maybe exploring what VR functionality might look like as opposed to just using a flat display on a PC monitor. Thanks for reading through this whole update! If you made it this far, I would really love to hear any feedback or questions on this or anything else. Most likely, it would help me figure out what some additional next steps would be, and I'd be super interested learn if there are other cool directions I could take this project!](https://youtu.be/zxoPFM5ol7o)
-
Help! Meet problems about connecting the ros with unity based on ros sharp
From my experience, ROS# is not the recommended way to connect Unity and ROS anymore. You probably want to use the ROS-TCP-Connector which is officially supported by Unity and its corresponding ROS package. That page has quite a few tutorials to help you get the connection setup. I currently maintain a plugin that implements a set of commonly used sensors here which might be of some use to you. Another useful resource maintained by Unity is their URDF importer, but it is still a bit buggy. Let me know if you need any other resources or help, I'm trying to make Unity a bit more livable for ROS developers.
ros-sharp
- Unity VS Unreal which one you recommand for ROS (am a biginner for all three)
-
Help! Meet problems about connecting the ros with unity based on ros sharp
User_App_ROS_UnitySimulationExample ยท siemens/ros-sharp Wiki (github.com)
-
Hello, I've been thinking about the same problem for 3 days.
RosSharp looks like can handle the visualization in Unity: https://github.com/siemens/ros-sharp/tree/master/Unity3D/Assets/RosSharp/Scripts/RosBridgeClient/SensorDataVisualization
-
Best approach to use ROS on windows.
For ROS1 - check out https://github.com/siemens/ros-sharp
-
Remote control of a a ROS machine from Windows?
The Microsoft ROS Landing page includes setup instructions for ROS natively on Windows. Additionally, if you are writing a Windows application, you can use ROS#, which includes .net bindings for ROS using the ROS Bridge.
-
We've created a open source Gazebo alternative that is much easier and more efficient to develop simulations.
The tools in the Unity Robotics Hub provide a URDF importer (derived from ROS# https://github.com/siemens/ros-sharp) and a low level TCP communications bridge to ROS.
-
Robotics on WSL2 using ROS, Docker and Unity 3D (Part II)
There are several ways of using Unity for robotics. Robotics for Unity is a new field and support is still custom made. Most of the applications out there are proprietary software created by private companies and not easily accessible or configurable by others. ROS support has been recently introduced by Unity itself, and it is a fork from a previous library from Siemens' RosSharp. In my setup, I used the Unity ROS library which works with the latest ArticulationBody types.
What are some alternatives?
Unity-Robotics-Hub - Central repository for tools, tutorials, resources, and documentation for robotics simulation in Unity.
Robotics-Object-Pose-Estimation - A complete end-to-end demonstration in which we collect training data in Unity and use that data to train a deep neural network to predict the pose of a cube. This model is then deployed in a simulated robotic pick-and-place task.
ros2_dotnet - .NET bindings for ROS2
ROS-TCP-Connector
unity_reachy_tutorial - Tutorial to run reachy on unity
URDF-Importer - URDF importer
DynamixelSDK - ROBOTIS Dynamixel SDK (Protocol1.0/2.0)
moveit - :robot: The MoveIt motion planning framework
gazebo-classic - Gazebo classic. For the latest version, see https://github.com/gazebosim/gz-sim
dynamixel-workbench - ROS packages for Dynamixel controllers, msgs, single_manager, toolbox, tutorials
dynamixel-workbench-msgs - Message package for dynamixel-workbench