stable-diffusion-webui VS U-2-Net

Music: The Prodigy - Smack My Bitch Up (Noisia Remix) Prompt: Giant sci-fi space machines made of realistic human eyes and snakes, snake eyes, tongues, beautiful, strange pupils, odd and intense colors, alien technology, robotic, organic, surreal, industrial, astronomy, floating in a vast bright dense amazing galaxy with lots of planets and stars, amazing high definition 4k beautiful detail. Params: Euler a, 20 steps, 20 scale, 0.55 denoise strength, seed 57420424 Method: - The input audio is an excerpt of The Prodigy - Smack My Bitch Up (Noisia Remix) - The input audio visualisation video was captured from https://butterchurnviz.com/, using "martin - witchcraft reloaded" as preset and just a dump screen recorder at ~60fps. - Input video reduced from 60fps to 20fps, so that processing time is reasonable, also helps reduce rapid flicker due to variations on every frame. - Each resulting was frame put through SD, mixed 50% with previously generated frame, to attempt to introduce some continuity. This was done using the Automatic1111 UI and a custom script based on Filarius's vid2vid script. - Resulting video pushed back up to 60fps using ffmpeg minterpolate.

is there a reason why "git clone https://github.com/Filarius/stable-diffusion-webui/blob/master/scripts/vid2vid.py" comes up with a "not found" error? I'm new to all this but was told the git pull system was the best way to grab things as you can update easily.

I use a finetuned custom Stable Diffusion model in combination with a style embedding for the characters for image generation and U²-Net for background removal.

Hi :) I am app/web developer and new to AI. Currently, I am making photo app which can segment all the things in image. I've used meta's segment anything. I've got all the masks but the boundary of masks are very bumpy. So I've tried rembg which uses u2net(salient object detection) and pymatting together. Do I have to use pymatting separately after getting segment from segment anything to improve boundary quality of my segmented output ?

Cool, thanks for sharing. It might be worth clearly attributing the models you're using, and maybe add a models/license file with the U2net license, since that license is different to the one you're using for your project, and since you're distributing the models.

Background Removal - I'd use u2net which has a model that's specifically trained on people vs backgrounds. If that didn't work, maybe DIS which is the newer version or rembg. These are pretty easy to get running I found.

u2net_human_seg (download, source): A pre-trained model for human segmentation.

Optimized – AI tweak Image background remover is now faster and enables trained model (onnx) swapping Revamp the python engine for background remover. Should be running faster than the previous build. Also added was the ability to replace the pre-trained ONNX model by the user themselves. https://github.com/xuebinqin/U-2-Net

-Added AI Background remover based on U2Net AI framework for Image projector. Check the Image projection "AI Tweaks" dropdown to toggle between u2net standard, u2netp – portrait, u2net-human_seg, u2net_cloth_seg, or silueta as the background remover AI model. As you can guess from the names, each of the models excels for different image subjects for background removal. For example, the portrait model is good for human portraits, cloth seg for clothing subjects and so on. Default to CUDA (Nvidia) processor, if you have an AMD card or would like to use CPU as the processor, untick the CUDA checkbox. Go here if you want to know more about the mechanics of U2Net -> https://github.com/xuebinqin/U-2-Net

Also checkout https://github.com/xuebinqin/U-2-Net. They have a new version in this repo: https://github.com/xuebinqin/DIS

There, the author has the same goal as you do, and has a train.py and instructions. You can reach out to the author and ask questions either in the issues section or perhaps email directly. Many times people are very helpful when you show interest in their work. The neural network it is based on (U2-net) is very easy to get running by the way, and has lots of use cases: https://github.com/xuebinqin/U-2-Net

really any segmentation model could work. "salient object detection" is well suited for "i have a single, obvious subject that I want to isolate from the background". This is the model I had in mind, but it wouldn't have to be this necessarily: https://github.com/xuebinqin/U-2-Net