DarkMark VS openpose

Also see DarkMark. For several years it has had support for loading custom Darknet/YOLO weights (not just MSCOCO!) to help annotate more images. https://www.ccoderun.ca/darkmark/Summary.html

You should look at DarkMark. I wrote it specifically to do what you describe. It is an annotation tool that loads the Darknet/YOLO weights, so it can assist in annotating images. I annotate a few images and train, reload DarkMark to annotate some more, train, rinse, lather, repeat.

Disclaimer: I'm the author of DarkHelp (the C++ library for Darknet) and DarkMark (the annotation and project management tool for Darknet).

If using Darknet/YOLO, look up DarkMark which does have support for video, as well as loading existing neural networks to help annotate images (or video frames) faster. Some info on getting started: https://www.ccoderun.ca/programming/darknet\_faq/#how\_to\_get\_started

You can find DarkMark here: https://github.com/stephanecharette/DarkMark

Disclaimer: I'm the author of DarkHelp and DarkMark, and I run the Darknet/YOLO discord.

I use DarkMark's video import functionality to extract video frames. See this screenshot: https://www.ccoderun.ca/darkmark/Summary.html#DarkMarkImportVideoFrames

DarkMark will extract frames from a video (lots of options, either all frames, sequences of frames, random number of frames, png vs jpeg, resize frames, ...) and then will let you annotate them as you normally would. https://github.com/stephanecharette/DarkMark

Open Pose, a multi-person keypoint detection library for body, face, hands, and foot estimation [10], is used for posing generated characters;

OpenPose

Perhaps something like OpenPose for pose estimation?

OpenPose looks like what you're looking for, it seems to have plugins for Unity. I can't say I've used it though.

You are correct. Open Pose has two keypoints for the eyes and two more for the ears. By saying were the ears are you automatically influence the angle of the head. You can see more about it on this github page. Just scroll a tiny bit and you can see a gif of the nodes overlapped on humans

FROM nvidia/cuda:10.0-cudnn7-devel-ubuntu18.04 # https://hub.docker.com/r/nvidia/cuda ENV DEBIAN_FRONTEND=noninteractive # install the dependencies for building OpenPose RUN apt-get update && # The rest is ignored for brevity. RUN pip3 install --no-cache-dir # The rest is ignored for brevity. # install cmake, clone OpenPose and download models RUN wget https://cmake.org/files/v3.20/cmake-3.20.2-linux-x86_64.tar.gz && \ # The rest is ignored for brevity. WORKDIR /openpose/build RUN alias python=python3 && cmake -DBUILD_PYTHON=OFF -DWITH_GTK=OFF -DUSE_CUDNN=ON .. # Build OpenPose. Cudnn 8 causes memory issues this is why we are using base with CUDA 10 and Cudnn 7 # Fix for CUDA 10.0 and Cudnn 7 based on the post below. # https://github.com/CMU-Perceptual-Computing-Lab/openpose/issues/1753#issuecomment-792431838 RUN sed -ie 's/set(AMPERE "80 86")/#&/g' ../cmake/Cuda.cmake && \ sed -ie 's/set(AMPERE "80 86")/#&/g' ../3rdparty/caffe/cmake/Cuda.cmake && \ make -j`nproc` && \ make install WORKDIR /openpose

One of the best cam only libraries (no depth sensor) I've seen is openpose, I ran it through a 360 camera and it was able to track body, face, and fingers really well even with spherical distortion from the 360 cam. example 360