Swin-Transformer-Semantic-Segmentation VS Video-Swin-Transformer

Semantic segmentation on ADE20K: https://github.com/SwinTransformer/Swin-Transformer-Semantic-Segmentation

PROBLEM STATEMENT Develop an efficient common strategy and relevant implementation to extract the video-based models in the black box and grey box setting across the following 2 problem statements. 1.Action Classification Model Extraction for Swin-T Model for Action Classification on Kinetics-400 dataset. Download the model from here- https://github.com/SwinTransformer/Video-Swin-Transformer 2.Video Classification Model Extraction for MoViNet-A2-Base Model for Video Classification on Kinetics- 600 dataset Download the model from here- https://tfhub.dev/tensorflow/movinet/a2/base/kinetics-600/classification/3 Blackbox Setting Do not use any relevant data set available and use synthetic or generated data without using the Kinetics series dataset. Also, do not use the same model architecture as the original model to train the extracted model. Greybox Setting You can use 5% of original data (balanced representation of classes) as a starting point to generate the attack dataset. Also, do not use the same model architecture as the original model to train the extracted model. Can someone explain the problem statement in a easy / understandable way ?? What I think is the models have already been provided and we have to do something in Blackbox and greybox . Can someone explain in brief what we have to do in the blackbox / greybox??

There are two main variants for the SWIN transformer the original SWIN transformer, official implementation here, and the Video SWIN transformer, official implementation here. Both architectures are very similar with the differences being mainly in the size of the input. The SWIN transformer pretrained on imagenet can be used as the backbone for different applications either image or video-based. In fact, the authors pretrained the original SWIN transformer on imagenet then they modified the input size and then fine-tuned it on video action recognition datasets. In your case, you can use the original SWIN transformer pretrained on imagenet then fine-tune it on your own dataset without modifying anything about the input size, since it is designed to process images.

The code is available on project GitHub. The paper Video Swin Transformer is on arXiv.