openseeface-gd VS OpenSeeFace

VSeeFace hasn't been made for Linux, but OpenSeeFace, the face tracker, is open-source and runs on linux using Python. And we use it to use other face tracking apps relying on OpenSeeFace, like Vtube Studio, Pupeteer / OpenSeeFace-GD

Contrary to what others are saying, you can load things from outside the res:// or user:// directories in an exported project. This project loads assets from a resources folder at runtime (and this is verifiable by the fact that assets can be modified after export).

This might be a option for me: https://github.com/you-win/openseeface-gd

It's not mine, exactly, but I got permission from the dev to package it into a flatpak. The dev themself stated that if it was easier to package it as a flatpak, they would do it themselves, so I am trying to make it so all they have to do is drag n drop the OSF-GD files in and build the flatpak. Link is https://github.com/you-win/openseeface-gd

OpenSeeFace-GD is an open source multi-platform Vtuber rendering program written in Godot. It has pre-compiled builds for Windows and Linux but is 100% compatible with Mac (you will have to build the project yourself in Godot, which is free, as you can't build for MacOS without a Mac and Youwin doesn't have a Mac).

I'm personally planning on integrating this into my project openseeface-gd in order to send scene data to OBS while having the controls open.

You can take a look at the codebase and see how the data is received and then applied

How long until some geeky girl in the free software community gives her a try and hooks her to the one, the only VTuber capture tool based on Godot? https://github.com/you-win/openseeface-gd

I've been working on OpenSeeFace GD for a few months now, and I can finally say I'm nearing completion in terms of usability.

I got something working modifying OpenSeeFace and it's an option and I might try to rewrite it in something compiled, but I'd like to look at the other options first.

Lastly, face tracking is either built in or uses a plugin device. You would also use OSC to manipulate blendshapes. I'd take a look at Opens Face.

Th best/only options I found were basically the old standby of Wine running closed source applications. Even then the end result was rather incomplete... because may of the best tracking options are simply closed source. (Note: Vseeface does offer an open source library)

There are only a few libraries that come to mind but take a bit of work to get started. MediaPipe Unity Plugin has eye tracking with a whole lot of types of tracking(head, hands, body). OpenSeeFace has models that do head and eye tracking. This repo uses Unity's neural net inference library, Barracuda, to run a MediaPipe iris landmark model (I haven't personally tested this library). Not sure how to translate eye landmarks to the position a player is looking at in a screen though. Hopefully this list of libraries gets you on the right path!

OpenSeeFace is open source. If you are using it with VSeeFace, you can just replace the Binary folder with your own build.

It uses OpenSeeFace for facetracking and engine patches/vrm code from the V-Sekai team.