tf-keras-vis VS pytorch-grad-cam

Hello! I'm trying to implement the Deep convolutional GAN of this paper: Weather GAN: Multi-Domain Weather Translation Using Generative Adversarial Networks by Xuelong Li, Kai Kou, Bin Zhao (arxiv) (architecture in image). The training data the authors used consists of images and correspondingsegmentation masks with labels 0-5 for each pixel (0 being no weather-relatedpixel). I have crudely made the segmentation module G(seg), Initialgenerator module G(init) and the Discriminator D, but I don'tunderstand how to do the attention module G(att). In the paper theymentioned they used pretained weights of VGG19, but very little else is saidabout G(att).I found this https://github.com/keisen/tf-keras-vislibrary, which might help me, as I guess I would want G(att) to extractsomething like the saliency or activation maps multiplied with the image.However, I don't know what kind of layers I should use, or what practicalitiesto use apart from the input and output. Should I transfer-learn the network withthis data, and if so, with the segmentation labels (i.e. if any label 1-5 ispresent in the attention pixel returned)? Or can I use the pre-trainedimagenet?Also, does anyone know if the layer colors in thearchitecture image mean anything, or if they are just selected randomly forvisualization? Especially I'm concerned why G(att)'s last encoderlayer (3rd layer) is colored differently from the first two. I was firstthinking that maybe it means that G(att) is a module inside G(seg),but apparently not. The three middle blocks are apparently residual blocks.

For the two examples we will be looking at, we will be using pytorch_grad_cam, an incredible open source package that makes working with GradCam very easy. There are excellent other tutorials to check out on the repo as well.

https://github.com/jacobgil/pytorch-grad-cam.

I know I would like to use grad-CAM https://github.com/jacobgil/pytorch-grad-cam

https://github.com/jacobgil/pytorch-grad-cam is a project that has a comprehensive collection of Pixel Attribution Methods for PyTorch (like the package name grad-cam that was the original algorithm implemented).