stylegan2-pytorch VS mediapipe

At lest, a way to complete the AI-generated cycle that https://thispersondoesnotexist.com/ started.

Thispersondoesnotexist.com - It's what the title says. It's terrifying but whenever you refrest the page, it shows you a new "person" who doesn't exist.

This X Does Not Exist is a list of gens for People, Cities, Words, Lyrics and other items that have been created by GANs (generative adversarial networks).

thispersondoesnotexist.com for making profile pics when you want to be anonymous

Nvidia is the one who made GANs popular. All this buzzword that people say about Generative AI, it was popularised by Nvidia with their StyleGAN paper. You must have seen or hear about thispersondoesnotexist.com? That's Nvidia.

GANs on the other hand uses a generator and discriminator model, where the generator learns to generate images that are "good enough" to fool the discriminator. It may make it easier to make discriminators that are better at picking out visual artefacts that stand out for humans, (it was easy to pick out "fake humans" from the early versions of thispersondoesnotexist.com by looking at their eyes and teeth, they tend to be pointing in different directions, or their teeth are too irregular before the discriminator got better at picking those out) but they can suffer from mode collapse, and generate images with very little variety.

Generative networks like StyleGAN

It's interesting that AI generated pictures of people can get most things right but seemingly always struggles with teeth & fingers. If you check out thispersondoesnotexist.com the teeth are almost always goofy looking...

Teeth as well, check out thispersondoesnotexist.com you'll come across a lot of convincing looking people with odd teeth

Today, we announce MediaPipe diffusion plugins, which enable controllable text-to-image generation to be run on-device. Expanding upon our prior work on GPU inference for on-device large generative models, we introduce new low-cost solutions for controllable text-to-image generation that can be plugged into existing diffusion models and their Low-Rank Adaptation (LoRA) variants.

You can just grab the TFLite version! https://github.com/google/mediapipe/blob/master/docs/solutions/models.md

I was going to say that I like the MediaPipe Selfie Segmentation model for doing this sort of thing in a web page, but I've just noticed (when getting the GitHub link[1]) that Google have marked the code as legacy[2] ... no idea if the new solution is better/easier to use[3].

For what it's worth, my CodePen using the old model is here: https://codepen.io/kaliedarik/pen/PopBxBM

[1] - https://github.com/google/mediapipe/blob/master/docs/solutio...

[2] - "Attention: Thank you for your interest in MediaPipe Solutions. As of April 4, 2023, this solution was upgraded to a new MediaPipe Solution."

[3] - https://developers.google.com/mediapipe/solutions/vision/ima...

I found MediaPipe's Face Mesh and was impressed with how simple it was to get going, but it just gives you the landmark points and I've not gone any further yet.

We've trained ControlNet on a subset of the LAION-Face dataset using modified output from MediaPipe's face mesh annotator to provide a new level of control when generating images of faces.

Medipaipe library link: https://mediapipe.dev/

If you're looking for the differences in terms of how inference is done, I recommend you take a look at MediaPipe's source code. MediaPipe doesn't use raw code, but uses a "graph" instead (eg. pose_landmark_cpu.pbtxt), which can be visualised using MediaPipe Viz. I also used axinc-ai/ailia-models as the base (preprocessing, inference, postprocessing, etc...) which I further built upon (keypoint refinement, roi from keypoints, filtering / smoothing, etc...)

I thought of doing that but unfortunately medipipe requires a RGB input and performs better with it more on that here