mediapipe VS Unity-Robotics-Hub

mediapipe/docs/solutions/pose.md at master · google/mediapipe · GitHub

Today, we announce MediaPipe diffusion plugins, which enable controllable text-to-image generation to be run on-device. Expanding upon our prior work on GPU inference for on-device large generative models, we introduce new low-cost solutions for controllable text-to-image generation that can be plugged into existing diffusion models and their Low-Rank Adaptation (LoRA) variants.

You can just grab the TFLite version! https://github.com/google/mediapipe/blob/master/docs/solutions/models.md

I believe Google already released transformers under an apache 2 license with a patent grant:

https://github.com/google/mediapipe/blob/master/mediapipe/mo...

I was going to say that I like the MediaPipe Selfie Segmentation model for doing this sort of thing in a web page, but I've just noticed (when getting the GitHub link[1]) that Google have marked the code as legacy[2] ... no idea if the new solution is better/easier to use[3].

For what it's worth, my CodePen using the old model is here: https://codepen.io/kaliedarik/pen/PopBxBM

[1] - https://github.com/google/mediapipe/blob/master/docs/solutio...

[2] - "Attention: Thank you for your interest in MediaPipe Solutions. As of April 4, 2023, this solution was upgraded to a new MediaPipe Solution."

[3] - https://developers.google.com/mediapipe/solutions/vision/ima...

I have used mediapipe very successfully in multiple projects and it's very easy to get running. You can choose from many different vision tasks including hand landmarks ( https://github.com/google/mediapipe/blob/master/docs/solutions/hands.md )

I found MediaPipe's Face Mesh and was impressed with how simple it was to get going, but it just gives you the landmark points and I've not gone any further yet.

We've trained ControlNet on a subset of the LAION-Face dataset using modified output from MediaPipe's face mesh annotator to provide a new level of control when generating images of faces.

Medipaipe library link: https://mediapipe.dev/

So far I have cloned the Universal_Robots_ROS2_Description and Robotiq_2f_140 (Supported for ROS2) in my workspace. Since I want to implement my Robot in Unity I am following the Unity Integration tutorial and the moveit tutorial.

Hi. We have used the ROS-TCP connector as well to establish a connection between ROS2 and Unity, and have done the ros_unity_integration tutorial.

It's for my thesis, so it's not a library and it's quite unstructured since there will be no further use. I used the MRTK Interface Elements in Unity and the Unity Robotics Hub (https://github.com/Unity-Technologies/Unity-Robotics-Hub). Then I customized everything for the KUKA iiwa Robotic Arm.

Actually, there is no code on GitHub yet. It's a Unity project sitting on my dropbox at the moment. I took the project in the Unity Robotics Hub repo (https://github.com/Unity-Technologies/Unity-Robotics-Hub.git) and added multiple things iteratively to make it work with the Hololens.

Are you using ROS for the robot by any chance? if so, have you seen this? it might do just what you want, incl way to connect between the robot and Unity: * https://github.com/Unity-Technologies/Unity-Robotics-Hub

How is this different from the the official unity for robotics? https://github.com/Unity-Technologies/Unity-Robotics-Hub