temporal-shift-module VS instruct-pix2pix

My question mostly has to do with optical flow. One of the two-stream networks I'm interested in trying out is TSN-TSM, as there are pre-trained weights available for it on the Assembly101 dataset released a few months ago.

This is the paper. (https://arxiv.org/abs/1811.08383). Here they are talking about how they can achieve temporal modelling by moving channels, which I assume are the RGB channels across frames. But I am super confused by the lingo. Here is the repo (https://github.com/mit-han-lab/temporal-shift-module). I can't give better rewards except virtual hugs. Thank you.

My guess is you're thinking of InstructPix2Pix[1], with prompts like "make the sky green" or "replace the fruits with cake"

[1] https://github.com/timothybrooks/instruct-pix2pix

This comes from https://github.com/timothybrooks/instruct-pix2pix, there is also an extension to use it in Automatic1111 Stable diffusion webui.

InstructPix2Pix

Is there another place I can download the model? I tried downloading the file using the instructions on this page:

It comes from https://github.com/timothybrooks/instruct-pix2pix, you can try it out https://huggingface.co/spaces/timbrooks/instruct-pix2pix

interesting, I've been running it using this: https://github.com/timothybrooks/instruct-pix2pix/blob/main/LICENSE

This is already done and it's already been implemented in the most popular web-ui for stable diffusion too. Granted the results aren't perfect yet.

Select the script, drop in a GIF, use img2img as normal to process it. Supports quick non-ffmpeg interpolation, and works surprisingly well with InstructPix2Pix. Intended to be a fun no-nonsense GIF pipeline.

Github Issue - Closed