mediapipe VS Linux-Fake-Background-Webcam

mediapipe/docs/solutions/pose.md at master · google/mediapipe · GitHub

Today, we announce MediaPipe diffusion plugins, which enable controllable text-to-image generation to be run on-device. Expanding upon our prior work on GPU inference for on-device large generative models, we introduce new low-cost solutions for controllable text-to-image generation that can be plugged into existing diffusion models and their Low-Rank Adaptation (LoRA) variants.

You can just grab the TFLite version! https://github.com/google/mediapipe/blob/master/docs/solutions/models.md

I believe Google already released transformers under an apache 2 license with a patent grant:

https://github.com/google/mediapipe/blob/master/mediapipe/mo...

I was going to say that I like the MediaPipe Selfie Segmentation model for doing this sort of thing in a web page, but I've just noticed (when getting the GitHub link[1]) that Google have marked the code as legacy[2] ... no idea if the new solution is better/easier to use[3].

For what it's worth, my CodePen using the old model is here: https://codepen.io/kaliedarik/pen/PopBxBM

[1] - https://github.com/google/mediapipe/blob/master/docs/solutio...

[2] - "Attention: Thank you for your interest in MediaPipe Solutions. As of April 4, 2023, this solution was upgraded to a new MediaPipe Solution."

[3] - https://developers.google.com/mediapipe/solutions/vision/ima...

I have used mediapipe very successfully in multiple projects and it's very easy to get running. You can choose from many different vision tasks including hand landmarks ( https://github.com/google/mediapipe/blob/master/docs/solutions/hands.md )

I found MediaPipe's Face Mesh and was impressed with how simple it was to get going, but it just gives you the landmark points and I've not gone any further yet.

We've trained ControlNet on a subset of the LAION-Face dataset using modified output from MediaPipe's face mesh annotator to provide a new level of control when generating images of faces.

Medipaipe library link: https://mediapipe.dev/

I'm trying to install Linux-Fake-Background-Webcam, which in turn requires 4l2loopback. Following the 4l2loopback manual, I try the following command to install it:

> Is this really a problem with Zoom or one with Wayland and the Linux distros?

I would say this is really a problem with Zoom, mostly due to the lack of attention paid to their web client.

For me, when it comes to video conferencing, the "table stakes" is working duplex audio over bluetooth, and so far only pipewire has solved this, which means Linux only.

After that, two very important features to have are screen sharing and background blur. For screen sharing, both Chrome/Chromium and Firefox have great support for screencast over pipewire, and so do many wayland compositors such as KDE kwin. As such, on a "pure" wayland system with no X11 window [1], screen sharing is also a solved problem.

Background blur is more ... complicated. When Google Meet rolled out https://ai.googleblog.com/2020/10/background-features-in-goo..., it immediately worked with chrome/chromium, on either wayland or X11. However, to this day, I think it still doesn't work with firefox, for whatever reason. Meanwhile, Zoom Linux client only started to support background blur (without green screen) since 5.7.6 released in August 2021, while Zoom web client remains hopeless.

In my previous job, we use Google Meet, and all is well with wayland. In the current one, we use Zoom, and I have to setup https://github.com/fangfufu/Linux-Fake-Background-Webcam as a workaround.

[1] This is very much doable these days, unless you want to run Slack desktop, which is hit-and-miss when running with --ozone-platform=wayland (unlike other electron-based apps such as Signal desktop), and of course the Zoom Linux app, which is just crap, as described in the article. Solution: run Slack and Zoom as Firefox tabs.

I was going to say, Zoom's absolutely broken greenscreen support is nicely hacked in X by the community with this excellent repo: https://github.com/fangfufu/Linux-Fake-Background-Webcam

It seems pretty cool, but why does it need to be run as root? I can run Linux-Fake-Background-Webcam just fine without root!

I use https://github.com/fangfufu/Linux-Fake-Background-Webcam and it works great (realtime) for video background removal.