MediaPipeUnityPlugin
OpenSeeFace
Our great sponsors
| MediaPipeUnityPlugin | OpenSeeFace | |
|---|---|---|
| 7 | 7 | |
| 1,597 | 1,312 | |
| - | - | |
| 8.9 | 4.2 | |
| 10 days ago | 3 months ago | |
| C# | Python | |
| MIT License | BSD 2-clause "Simplified" License |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
MediaPipeUnityPlugin
- any hand detect api or plugin that I can use in arfoundation?
-
Unity Visual Scripting and Hand Tracking
Look into Playmaker and Mediapipe.
-
MediaPipe in Unity
You should really specify where you got stuck. In any event, there are compiled packages here https://github.com/homuler/MediaPipeUnityPlugin/releases YMMV.
-
Been working on a VTuber App
This person has ported the majority of the C++ API to Unity's C#, Github link here. Assuming they've ported it one-to-one, you'd be using the same API I use, just in C# (I've not tried it though)
-
When I call it "AR Experience"?
Also, there is a MedeaPipeUnityPlugin.
-
Open map with gaze tracking for someone with paralysis
There are only a few libraries that come to mind but take a bit of work to get started. MediaPipe Unity Plugin has eye tracking with a whole lot of types of tracking(head, hands, body). OpenSeeFace has models that do head and eye tracking. This repo uses Unity's neural net inference library, Barracuda, to run a MediaPipe iris landmark model (I haven't personally tested this library). Not sure how to translate eye landmarks to the position a player is looking at in a screen though. Hopefully this list of libraries gets you on the right path!
-
3D Realtime Pose Detection with MediaPipe and Unity3D.
Checkout https://github.com/homuler/MediaPipeUnityPlugin
OpenSeeFace
-
Getting face feature pose statistics
I got something working modifying OpenSeeFace and it's an option and I might try to rewrite it in something compiled, but I'd like to look at the other options first.
-
This may be a silly question but can I hire someone to make me a customized avatar for vr chat?
Lastly, face tracking is either built in or uses a plugin device. You would also use OSC to manipulate blendshapes. I'd take a look at Opens Face.
-
Any recommendations for VTuber setup on Linux? Ideally something that's completely open source.
Th best/only options I found were basically the old standby of Wine running closed source applications. Even then the end result was rather incomplete... because may of the best tracking options are simply closed source. (Note: Vseeface does offer an open source library)
- Running OpenSeeFace on Linux with python 3.10
-
Open map with gaze tracking for someone with paralysis
There are only a few libraries that come to mind but take a bit of work to get started. MediaPipe Unity Plugin has eye tracking with a whole lot of types of tracking(head, hands, body). OpenSeeFace has models that do head and eye tracking. This repo uses Unity's neural net inference library, Barracuda, to run a MediaPipe iris landmark model (I haven't personally tested this library). Not sure how to translate eye landmarks to the position a player is looking at in a screen though. Hopefully this list of libraries gets you on the right path!
-
I'm going to try to use VSeeFace again, is there a way to change how lip syncing works?
OpenSeeFace is open source. If you are using it with VSeeFace, you can just replace the Binary folder with your own build.
-
I'm making a renderer for facetracking data
It uses OpenSeeFace for facetracking and engine patches/vrm code from the V-Sekai team.
What are some alternatives?
AssetStudio - AssetStudio is a tool for exploring, extracting and exporting assets and assetbundles.
openseeface-gd - A GUI for running OpenSeeFace.
ILSpy - .NET Decompiler with support for PDB generation, ReadyToRun, Metadata (&more) - cross-platform!
kalidokit - Blendshape and kinematics calculator for Mediapipe/Tensorflow.js Face, Eyes, Pose, and Finger tracking models.
Entitas - Entitas is a super fast Entity Component System (ECS) Framework specifically made for C# and Unity
UniVRM - UniVRM is a gltf-based VRM format implementation for Unity. English is here https://vrm.dev/en/ . 日本語 はこちら https://vrm.dev/
ml-agents - The Unity Machine Learning Agents Toolkit (ML-Agents) is an open-source project that enables games and simulations to serve as environments for training intelligent agents using deep reinforcement learning and imitation learning.
vpuppr - VTuber application made with Godot 4
lively - Free and open-source software that allows users to set animated desktop wallpapers and screensavers powered by WinUI 3.
VTuber_Unity - Use Unity 3D character and Python deep learning algorithms to stream as a VTuber!
VRigUnity - VRM hand tracking using mediapipe
fastT5 - ⚡ boost inference speed of T5 models by 5x & reduce the model size by 3x.