openpose VS freemocap

Open Pose, a multi-person keypoint detection library for body, face, hands, and foot estimation [10], is used for posing generated characters;

OpenPose

Perhaps something like OpenPose for pose estimation?

OpenPose looks like what you're looking for, it seems to have plugins for Unity. I can't say I've used it though.

You are correct. Open Pose has two keypoints for the eyes and two more for the ears. By saying were the ears are you automatically influence the angle of the head. You can see more about it on this github page. Just scroll a tiny bit and you can see a gif of the nodes overlapped on humans

FROM nvidia/cuda:10.0-cudnn7-devel-ubuntu18.04 # https://hub.docker.com/r/nvidia/cuda ENV DEBIAN_FRONTEND=noninteractive # install the dependencies for building OpenPose RUN apt-get update && # The rest is ignored for brevity. RUN pip3 install --no-cache-dir # The rest is ignored for brevity. # install cmake, clone OpenPose and download models RUN wget https://cmake.org/files/v3.20/cmake-3.20.2-linux-x86_64.tar.gz && \ # The rest is ignored for brevity. WORKDIR /openpose/build RUN alias python=python3 && cmake -DBUILD_PYTHON=OFF -DWITH_GTK=OFF -DUSE_CUDNN=ON .. # Build OpenPose. Cudnn 8 causes memory issues this is why we are using base with CUDA 10 and Cudnn 7 # Fix for CUDA 10.0 and Cudnn 7 based on the post below. # https://github.com/CMU-Perceptual-Computing-Lab/openpose/issues/1753#issuecomment-792431838 RUN sed -ie 's/set(AMPERE "80 86")/#&/g' ../cmake/Cuda.cmake && \ sed -ie 's/set(AMPERE "80 86")/#&/g' ../3rdparty/caffe/cmake/Cuda.cmake && \ make -j`nproc` && \ make install WORKDIR /openpose

One of the best cam only libraries (no depth sensor) I've seen is openpose, I ran it through a 360 camera and it was able to track body, face, and fingers really well even with spherical distortion from the 360 cam. example 360

You could try with https://freemocap.org/ before to invest in a Rokoko

I didn't try it yet, but maybe it could be useful for you https://freemocap.org/

There's also FreeMoCap, a markerless mocap solution that you can run using just couple of cheap webcams. It's also a non-profit research project with the most open 'copyleft' license you can find.

FreeMoCap (nvm, requires Charuco board)

Thank you for your hard work! I thenk the most promising work right now is this: FreeMoCap not only tracks balls and hand, but the whole body too. I know the approach is different as it can only work by multiple view camera with a static tracker, but Jon (the creator of it) made also a Discord and people are actively giving suggestions, building tools (tike to transfer the recorded position data to maki rigging ik Blender) so i think more than the the projects of hundred of people alone doing the same thing every time, the fact that there is a community pushing forward the same project is so great.

I would take a look at the FreeMoCap project. It's still in development and very rough around the edges, but they're working towards a more user-friendly version in hopefully the next few months.

Markerless motion capture has been developed various times in open source projects. Here's one that allows you to record motion and import it into blender: https://github.com/jonmatthis/freemocap, and the subreddit: r/FreeMoCap