kalidokit VS animegan2-pytorch

I try to animate a ReadyPlayer Me avatar using ThreeJS and Kalidokit (or something else) with MediaPipe Hollisitc Pose. Here is a working JSFiddle :

kalidoface is made by /u/YeeMachineDev and seems to be fully open source https://github.com/yeemachine/kalidokit https://github.com/yeemachine/kalidoface-3d

In terms of software, what you linked used optical tracking. The underlying tech is commonly called machine/computer vision. You can develop your own tracking software, but there's some open source solutions, like OpenCV than make it faster to build apps. (You can use OpenCV with Python bindings to learn the basics somewhat quickly). What you linked uses Kalidokit, an open source solution using TensorFlow.js and Google's MediaPipe.

oh my apologies! may I suggest Kalidoface then? https://kalidoface.com/ free face and arm tracking! plus they have a very friendly discord for troubleshooting

[JS newbie here] I just discovered kalidokit, a vtuber JS app by yeemachine, and I'm trying to figure out how I could play with its look and feel. Do you have any recommendations on how to proceed? In short, I don't know what are the relevant files in the repo, how they are connected and how to run the app on a local machine. Any help would be very helpful! Thanks.

kalidokit

rom PIL import Image import torch import IPython from IPython.display import display # upload images from google.colab import files uploaded = files.upload() # https://github.com/bryandlee/animegan2-pytorch # load models and face2paint utility function model_facev2 = torch.hub.load("bryandlee/animegan2-pytorch:main", "generator", pretrained="face_paint_512_v2") face2paint = torch.hub.load("bryandlee/animegan2-pytorch:main", "face2paint", size=512) for INPUT_IMG in ['KatLi.JPG']: img = Image.open(INPUT_IMG).convert("RGB") out_facev2 = face2paint(model_facev2, img) # display images display(img) display(out_facev2)

I'll point you towards Prism, then. As far as style transfer goes, it's the best I've found so far. Extremely simple to run in colab and offline, plus the results are quite good for what it's doing. I used it a ton before I started running Face Portrait v2 offline as a general "art" filter for things I'm working on. It does stylization and some facial morphing in one pass, which adds a lot more personality than style transfer alone. You can also use it on anything, not just faces.

I would recommend https://github.com/bryandlee/animegan2-pytorch and have used it for other videos, but I’m still working on my own model, the results of which you see above 👆🏻

Sure, I used this Python library and the Inshot video editor https://github.com/bryandlee/animegan2-pytorch

PyTorch Implementation of AnimeGANv2

(2021.02.21) The pytorch version of AnimeGANv2 has been released, Be grateful to @bryandlee for his contribution.

AnimeGAN2 PyTorch

github: https://github.com/bryandlee/animegan2-pytorch

source tweet: https://twitter.com/Yokohara\_h/status/1466521442686685188?s=20 web app(webcam): https://huggingface.co/spaces/nateraw/animegan-v2-for-videos web app(images): https://huggingface.co/spaces/akhaliq/AnimeGANv2 github: https://github.com/bryandlee/animegan2-pytorch