RobustVideoMatting
stable-diffusion-webui-depthmap-script
RobustVideoMatting | stable-diffusion-webui-depthmap-script | |
---|---|---|
16 | 64 | |
8,189 | 1,587 | |
- | - | |
0.0 | 8.3 | |
about 1 month ago | about 2 months ago | |
Python | Python | |
GNU General Public License v3.0 only | MIT License |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
RobustVideoMatting
- lineart_coarse + openpose, batch img2img
-
Tools For AI Animation and Filmmaking , Community Rules, ect. (**FAQ**)
Robust Video Matting/Background Remover (Remove Background from images and videos, useful for compositing) https://github.com/PeterL1n/RobustVideoMatting (RVM - Remove backgrounds from videos) https://github.com/nadermx/backgroundremover (BackgroundRemover - works well on single images) -------VOICE GENERATION--------
-
Adobe After Effects VS Runway AI 👀
Looks like runway is packaging a bunch of AI tools like stable diffusion and other opensource tools into a paid package. The matting tools it is using looks like this tool https://github.com/PeterL1n/RobustVideoMatting which can be run off your computer for free if you can figure out the geeky side of installing this stuff. I've tried it out and it sometimes works well but most of the time the results aren't as good as the examples on their github. Still a good tool to have in the toolbox though.
-
Rotoscoping a video by comparing images
OR this separate application looks promising, if you can work out Google Collab (I couldn't unfortunately): https://github.com/PeterL1n/BackgroundMattingV2 https://github.com/PeterL1n/RobustVideoMatting
-
CatFileCreator in Nuke
I have done a bit of coding and I will use pretrained models only. Looking at things like depth and segmentation. Like this as an example. I am using it on a collab now but its so cumbersome. https://github.com/PeterL1n/RobustVideoMatting
-
[Q] Video Editing using AI
I do not know much about Machine learning, and I am not sure if I can ask question here. But if yes, I need help with either choosing best libraries to do Video Editing like Background Removal and similar. Some of the ones that I found is RVM: https://github.com/PeterL1n/RobustVideoMatting (which currently seems like the best choice)
- Is this FOSS ML software safe?
- [D] Is this ML project safe?
-
Trying to train videomatting model
First of all I would ask if somebody retrained Robust Video Matting model on own data? I am trying to, but with all the models I end up getting bad quality result as the ones attached to the post. So my data is some objects rotating on 360 and with white backgrounds, The task seems to be pretty simple as the model just has to remove white bgr and keep colorized object. I have masks on every 10th frame of my videos. The masks are 0 - bgr, 255 - fgr. I have tried Robust Video Matting model, MODNet, PaddleSeg and several segmentation models and every of them failed to show consistent results on that data. What should I do in the case?
-
Remove Background NO GREENSCREEN?
I have found a github with a project like this but it is tedious to use: https://github.com/PeterL1n/RobustVideoMatting
stable-diffusion-webui-depthmap-script
-
PATCHFUSION is really impressive. High resolution depth maps in 16bit. I've been waiting for this. https://github.com/zhyever/PatchFusion
The guide on the github page for the extension is OK: https://github.com/thygate/stable-diffusion-webui-depthmap-script
-
Extension not showing. Depthmap help 🙏
New to SD. I'm trying to get an extension to work (https://github.com/thygate/stable-diffusion-webui-depthmap-script) but opposite to the tutorials the "depth" tab doesn't show after installation. Anyone who can help locate the problem? Thanks!
-
Is anyone working on stereoscopic 3D SD? Is it even possible?
You can use this extension to generate stereoscopic images . . . I don't (yet) dabble in video, so I don't know what it'll do there. I've done a ton of stereo pics with it. My fascination sort of comes and goes. You can do cross-eyed or parllel view as well as red/cyan anaglyphs.
-
GUIDE: Ways to generate consistent environments for comics, novels, etc
Option 8. Use img2img of existing 360 HDRIs, extract their depth maps with the depth extension. Use that as a displacement map on a sphere in Blender, similarly to this, with the refurbished HDRI as an image texture, then take screenshots from a position close to the center of the sphere. You are limited to staying close to the center in order to avoid distortion, but now you have 360 degrees of consistent freedom for a particular scene. If you have 2 or more HDRIs of the same place, even better. You could also combine this with the 3D environments of the other options to use 360 renders as bases for the img2img.
-
Another Ai image to 3d
you have another automatic 1111 extension that allow you to create there also the 3d file, but this consume a lot of vram https://github.com/thygate/stable-diffusion-webui-depthmap-script
-
Get a 16-Bit Controlnet Depth
If you're using A1111 webui there is the depthmap2mask extension which you can install from the extensions tab. It will add a depth tab which will allow you to create 16-bit depth maps among many other things.
-
180 VR - Blue Techno World - (Stable Diffusion + Deforum) stereo video
actually it is very easy to do. What you need is install extension for Stable Diffusion webUI (https://stable-diffusion-art.com/install-windows/) . This extension will generate stereo for you automatically. Name is Depth. (https://github.com/thygate/stable-diffusion-webui-depthmap-script)
- Is it possible for me to approximate a depth map from a generated image and make a 3D model?
- Thanks for loving our Star Wars video! We created a new one for Lord of the Rings. Enjoy this mid-journey to Middle-Earth.
-
Found this site through twitter that slightly animates images. Throwing Stable Diffusion generations into it is pretty awesome. Site in comments.
You can do this inside of a1111 as well with this extension https://github.com/thygate/stable-diffusion-webui-depthmap-script
What are some alternatives?
MODNet - A Trimap-Free Portrait Matting Solution in Real Time [AAAI 2022]
MiDaS - Code for robust monocular depth estimation described in "Ranftl et. al., Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross-dataset Transfer, TPAMI 2022"
BackgroundMattingV2 - Real-Time High-Resolution Background Matting
a1111-sd-zoe-depth - a1111 sd WebUI extention version of ZoeDepth
PINTO_model_zoo - A repository for storing models that have been inter-converted between various frameworks. Supported frameworks are TensorFlow, PyTorch, ONNX, OpenVINO, TFJS, TFTRT, TensorFlowLite (Float32/16/INT8), EdgeTPU, CoreML.
multi-subject-render - Generate multiple complex subjects all at once!
pytorch-deep-image-matting - Pytorch implementation of deep image matting
Thin-Plate-Spline-Motion-Model - [CVPR 2022] Thin-Plate Spline Motion Model for Image Animation.
coremltools - Core ML tools contain supporting tools for Core ML model conversion, editing, and validation.
depthmap2mask - Create masks out of depthmaps in img2img
keras-onnx - Convert tf.keras/Keras models to ONNX
point-e - Point cloud diffusion for 3D model synthesis