RobustVideoMatting
Thin-Plate-Spline-Motion-Model
RobustVideoMatting | Thin-Plate-Spline-Motion-Model | |
---|---|---|
16 | 28 | |
8,176 | 3,297 | |
- | - | |
0.0 | 1.9 | |
30 days ago | 3 months ago | |
Python | Jupyter Notebook | |
GNU General Public License v3.0 only | MIT License |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
RobustVideoMatting
- lineart_coarse + openpose, batch img2img
-
Tools For AI Animation and Filmmaking , Community Rules, ect. (**FAQ**)
Robust Video Matting/Background Remover (Remove Background from images and videos, useful for compositing) https://github.com/PeterL1n/RobustVideoMatting (RVM - Remove backgrounds from videos) https://github.com/nadermx/backgroundremover (BackgroundRemover - works well on single images) -------VOICE GENERATION--------
-
Adobe After Effects VS Runway AI 👀
Looks like runway is packaging a bunch of AI tools like stable diffusion and other opensource tools into a paid package. The matting tools it is using looks like this tool https://github.com/PeterL1n/RobustVideoMatting which can be run off your computer for free if you can figure out the geeky side of installing this stuff. I've tried it out and it sometimes works well but most of the time the results aren't as good as the examples on their github. Still a good tool to have in the toolbox though.
-
Rotoscoping a video by comparing images
OR this separate application looks promising, if you can work out Google Collab (I couldn't unfortunately): https://github.com/PeterL1n/BackgroundMattingV2 https://github.com/PeterL1n/RobustVideoMatting
-
CatFileCreator in Nuke
I have done a bit of coding and I will use pretrained models only. Looking at things like depth and segmentation. Like this as an example. I am using it on a collab now but its so cumbersome. https://github.com/PeterL1n/RobustVideoMatting
-
[Q] Video Editing using AI
I do not know much about Machine learning, and I am not sure if I can ask question here. But if yes, I need help with either choosing best libraries to do Video Editing like Background Removal and similar. Some of the ones that I found is RVM: https://github.com/PeterL1n/RobustVideoMatting (which currently seems like the best choice)
- Is this FOSS ML software safe?
- [D] Is this ML project safe?
-
Trying to train videomatting model
First of all I would ask if somebody retrained Robust Video Matting model on own data? I am trying to, but with all the models I end up getting bad quality result as the ones attached to the post. So my data is some objects rotating on 360 and with white backgrounds, The task seems to be pretty simple as the model just has to remove white bgr and keep colorized object. I have masks on every 10th frame of my videos. The masks are 0 - bgr, 255 - fgr. I have tried Robust Video Matting model, MODNet, PaddleSeg and several segmentation models and every of them failed to show consistent results on that data. What should I do in the case?
-
Remove Background NO GREENSCREEN?
I have found a github with a project like this but it is tedious to use: https://github.com/PeterL1n/RobustVideoMatting
Thin-Plate-Spline-Motion-Model
-
Okay, that's Ai but how?
Exactly what I was thinking, looks a lot like thin plate spline motion, maybe with some layers/composition for the wings and hair.
- Is it possible to sync a lip and facial expression animation with audio in real time?
- GitHub - yoyo-nb/Thin-Plate-Spline-Motion-Model: [CVPR 2022] Thin-Plate Spline Motion Model for Image Animation. (Question: How do I increase the resolution of the output?)
-
Tools For AI Animation and Filmmaking , Community Rules, ect. (**FAQ**)
First Order Motion Model/Thin Plate Spline (Animate Single images realistically using a driving video) https://github.com/AliaksandrSiarohin/first-order-model (FOMM - Animate still images using driving videos) https://github.com/yoyo-nb/Thin-Plate-Spline-Motion-Model (Thin Plate Spline - Likely just a repost of FOMM but with better documentation and tutorials on YouTube) https://drive.google.com/drive/folders/1PyQJmkdCsAkOYwUyaj_l-l0as-iLDgeH (FOMM/Thin Plate Checkpoints) https://disk.yandex.com/d/lEw8uRm140L_eQ (FOMM/Thin Plate Checkpoints mirror) -------3D ANIMATION--------
-
Help from Community [Development]
GitHub - yoyo-nb/Thin-Plate-Spline-Motion-Model: [CVPR 2022] Thin-Plate Spline Motion Model for Image Animation.
- Elvis & James Blunt singing together - doing Elvis voice synthesis & using Thin-Plate-Spline model for a cheap fast deepfake video to sync
- Does Anyone Know What Tool is used to Make These TikTok Videos?
-
How did he do this?
Probably with using this https://github.com/yoyo-nb/Thin-Plate-Spline-Motion-Model
-
Animate your stable diffusion portraits
Use https://github.com/yoyo-nb/Thin-Plate-Spline-Motion-ModelHuggingface demo: https://huggingface.co/spaces/CVPR/Image-Animation-using-Thin-Plate-Spline-Motion-ModelGoogle Colab: https://colab.research.google.com/drive/1DREfdpnaBhqISg0fuQlAAIwyGVn1loH_?usp=sharing
- SD + thin-plate-spline-motion-model
What are some alternatives?
MODNet - A Trimap-Free Portrait Matting Solution in Real Time [AAAI 2022]
first-order-model - This repository contains the source code for the paper First Order Motion Model for Image Animation
BackgroundMattingV2 - Real-Time High-Resolution Background Matting
Wav2Lip - This repository contains the codes of "A Lip Sync Expert Is All You Need for Speech to Lip Generation In the Wild", published at ACM Multimedia 2020. For HD commercial model, please try out Sync Labs
PINTO_model_zoo - A repository for storing models that have been inter-converted between various frameworks. Supported frameworks are TensorFlow, PyTorch, ONNX, OpenVINO, TFJS, TFTRT, TensorFlowLite (Float32/16/INT8), EdgeTPU, CoreML.
DFL-Colab - DeepFaceLab fork which provides IPython Notebook to use DFL with Google Colab
pytorch-deep-image-matting - Pytorch implementation of deep image matting
SadTalker - [CVPR 2023] SadTalker:Learning Realistic 3D Motion Coefficients for Stylized Audio-Driven Single Image Talking Face Animation
coremltools - Core ML tools contain supporting tools for Core ML model conversion, editing, and validation.
articulated-animation - Code for Motion Representations for Articulated Animation paper
keras-onnx - Convert tf.keras/Keras models to ONNX
stable-diffusion-webui-depthmap-script - High Resolution Depth Maps for Stable Diffusion WebUI