ECCV2022-RIFE
Anime4K
ECCV2022-RIFE | Anime4K | |
---|---|---|
12 | 88 | |
4,090 | 17,825 | |
2.0% | - | |
5.8 | 3.5 | |
2 months ago | 11 days ago | |
Python | Jupyter Notebook | |
MIT License | MIT License |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
ECCV2022-RIFE
-
AI Frame interpolation Question
Check out RIFE.
-
Enhancing ControlNet-m2m Video Smoothness with Multi-Level Frame Interpolation
Using Flowframes with the RIFE model, run 2x interpolation on a folder of video frames.
-
New NVIDIA Driver with RTX Video Super Resolution is Now Available!
Personally I have mine set to use RIFE AI via TensorRT for frame interpolation(x2), if the FPS is 30 or less.
-
I just added ControlNet BATCH support in automatic1111 webui and ControlNet extension, and here's the result. Read comment to support the Pull Requests so you can use this technique as soon as possible.
Oh now that I saw this comment, I started to investigate frame interpolation techniques using AI and found this: https://github.com/megvii-research/ECCV2022-RIFE
-
How can indie devs make 2d animations quickly, or streamline the process?
Yes but you need to use a different AI first. There are multiple AI like RIFE (there are apps for it if you don't like code) that will smooth out your animation. Then you can use those frames with Novel AI to get a more organic look in the end.
-
ECCV2022-RIFE VS FluidFrames.RIFE - a user suggested alternative
2 projects | 4 Feb 2023
-
Inpainting every frame using AE + SD
To have more smooth effect you can reduce frame per second and add FILM or RIFE between frames.
-
I inserted myself into stable diffusion, not perfect but it kinda looks my face
Interpolated with https://github.com/megvii-research/ECCV2022-RIFE
-
Stable Diffusion Animation
Sure! This would be my approach (and tools) if I was smarter:
If you make the generations with some similarities and use the right interpolation, you don't need 1000 images like my video and can obtain a smooth movement.
First, generate images with some kind of visual anchor (background, an object). You can use frames generated using the previous frame as reference image, or the same seed but different prompt/parameters, or you can go wild using img2img/inpainting (btw I struggle to find an inpainting tool for Stable Diffusion: they seem to be just img2img with a mask, without contest).
Then pass the generated images to one of the most recent interpolation algorithms, like this one https://github.com/megvii-research/ECCV2022-RIFE or the one used in the replicate we are commenting on (someone posted this reference: https://github.com/google-research/frame-interpolation )
The first link reports some free and paid implementation and a Colab, so depending on how deep you want to go, you have a lot of choices.
In the end, I'd use some good app to stabilize the image if needed, to get a more "calm" look. I use Luma Fusion, but it's a paid app (cheap, one-time payment, for iOS). I'm sure there are a ton of open-source implementations.
It's an approach similar to the animation on replicate, but it allows a lot of fine-tuning and you can add new animation ideas/tools to the process.
Nothing revolutionary, but I hope it helps!
> You have generated some pretty cool designs.
Thanks! I put in a lot of work in the last weeks. The project has a mission, I wrote something, but it's not ready yet. I believe it will be with the launch of Dall-E 8 :-/
-
Help with interpolating "missing" frames from source video
You'd probably get way better results by using something like RIFE to do interpolation and recreate missing frames, instead of minterpolate. I understand though that it's more effort as you'll need to install and setup RIFE.
Anime4K
-
AI Made These Movies Sharper. Critics Say It Ruined Them.
I watch AI enhanced movies and/or shows every day when I use Anime4K, a real-time ML upscaler for anime https://github.com/bloc97/Anime4K
If you give shit-tier blurry DVD messes, it turns them into nicer blurry messes.
If you give it the usual bitrate starved streaming quality show, it looks almost as nice as a blueray.
If you give it a blueray... It basically comes out looking exactly the same. I'll usually still add its "darken lines" filter since I'm a fan of strong linework.
-
IPTV Anti-Piracy Threats May Increase Male Motivation to Pirate by 30%
> Can you explain what mpv offers over vlc ?
People have complained about VLC's decoding quality in the past (no specific source but lots of google results). I'm sure there's valid bugs that have been patched over and some still there, but I'd take it with a grain of salt since VLC is super popular and therefore more noise when it drops the ball.
That said, there's a few MPV-only features I absolutely love. One is that you can display multiple subtitles at once (for learning foreign languages) and another is that you can drop in upscaling / sharpening filters like [1].
[1] https://github.com/bloc97/Anime4K
-
AI-Powered Nvidia RTX Video HDR Transforms Standard Video into HDR Video
If you want to avoid manual processing, Anime4K runs in real time as a GLSL shader you can load into MPV (or Plex or something called IINA according to the readme) and still gives great results.
https://github.com/bloc97/Anime4K
- What software to use for upscaling anime edits
-
Load shader in Android MPV?
I came up with some shaders (here) that I want to load on the Android MPV. I copied the shaders in my Android phone. According to what I found, I edited the mpv.conf with the locations of the shaders as below:
-
They Bypass YouTube Video Download Throttling
Players like VLC do much better and let you make your own arbitrary adjustments via GLSL :)
In particular, I like the anime4k shader pack, which is ML-based but runs in real time in mpv (and I think VLC as well). While it is tuned for anime (as is obvious from the name), it has decent denoise and deblur which often make YT content more watchable and a restore step that does a really good job with compression artifacts but is a bit too tuned for anime so may not always work, or even make things worse. See https://github.com/bloc97/Anime4K/releases
-
Pls help me a find an anime website
I know of https://github.com/bloc97/Anime4K that can do real-time upscaling of anime.
-
[Shaders] Les shaders haut de gamme AI dans le lecteur vidéo MPV en temps réel [Anime 4K, FSRCNNX]
Anime 4k
-
NVIDIA RTX Video Super Resolution is now supported by VLC media player - VideoCardz.com
You can start with the Anime4k setup guide: https://github.com/bloc97/Anime4K/blob/master/md/GLSL_Instructions_Windows.md
-
Microsoft Edge is getting a video upscaler to make blurry old videos look better
Personally, it's not really any better than mpv filters like NIS [0] (Nvidia Image Scaling, which does not use AI but is somehow more sharp than these filters) as well as Anime4k [1] (which does use AI but only in the training phase to create the initial glsl file which then does not use AI inferencing but plain old image filters).
Combine mpv with its yt-dlp [2] and SponsorBlock support [3] and you have a competent way to upscale your own YouTube (and other types of [4]) videos.
[0] https://gist.github.com/agyild/7e8951915b2bf24526a9343d951db...
[1] https://github.com/bloc97/Anime4K
[2] https://www.funkyspacemonkey.com/replace-youtube-dl-with-yt-...
[3] https://github.com/po5/mpv_sponsorblock
[4] https://github.com/yt-dlp/yt-dlp/blob/master/supportedsites....
What are some alternatives?
stable-diffusion-webui - Stable Diffusion web UI
waifu2x - Image Super-Resolution for Anime-Style Art
frame-interpolation - FILM: Frame Interpolation for Large Motion, In ECCV 2022.
better-mpv-config - An improved MPV Media Player v3 configuration file.
sd-webui-controlnet - WebUI extension for ControlNet
FSRCNN-TensorFlow - An implementation of the Fast Super-Resolution Convolutional Neural Network in TensorFlow
arXiv2021-RIFE - Real-Time Intermediate Flow Estimation for Video Frame Interpolation [Moved to: https://github.com/hzwer/ECCV2022-RIFE]
plex-mpv-shim - Cast media from Plex Mobile and Web apps to MPV. (Unofficial)
VideoRenderer - RTX HDR modded into MPC-VideoRenderer.
iina - The modern video player for macOS.
txt2mask - Automatically create masks for Stable Diffusion inpainting using natural language.
mpv - 🎥 Command line video player