pytorch-grad-cam
Transformer-Explainability
Our great sponsors
pytorch-grad-cam | Transformer-Explainability | |
---|---|---|
5 | 1 | |
9,351 | 1,656 | |
- | - | |
5.4 | 0.0 | |
about 1 month ago | 3 months ago | |
Python | Jupyter Notebook | |
MIT License | MIT License |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
pytorch-grad-cam
-
Exploring GradCam and More with FiftyOne
For the two examples we will be looking at, we will be using pytorch_grad_cam, an incredible open source package that makes working with GradCam very easy. There are excellent other tutorials to check out on the repo as well.
-
Which layers are doing image segmentation on AutoEncoders/U-NET?
https://github.com/jacobgil/pytorch-grad-cam.
-
[D] Algorithm for view prediction?
I know I would like to use grad-CAM https://github.com/jacobgil/pytorch-grad-cam
-
[P] Adapting Class Activation Maps for Object Detection and Semantic Segmentation
https://github.com/jacobgil/pytorch-grad-cam is a project that has a comprehensive collection of Pixel Attribution Methods for PyTorch (like the package name grad-cam that was the original algorithm implemented).
- [Project] Recent Class Activation Map Methods for CNNs and Vision Transformers
Transformer-Explainability
-
[Project] Recent Class Activation Map Methods for CNNs and Vision Transformers
Not exactly the same but since you mentioned using ViT's attention outputs as a 2D feature map for the CAM you can consider this paper (Transformer Interpretability Beyond Attention Visualization) where they study the question of how to choose/mix the attention scores in a way that can be visualized (so similar to the CAMs). Maybe it can lead to better results. https://arxiv.org/abs/2012.09838 https://github.com/hila-chefer/Transformer-Explainability
What are some alternatives?
pytorch-lightning - Build high-performance AI models with PyTorch Lightning (organized PyTorch). Deploy models with Lightning Apps (organized Python to build end-to-end ML systems). [Moved to: https://github.com/Lightning-AI/lightning]
shap - A game theoretic approach to explain the output of any machine learning model.
pytorch-CycleGAN-and-pix2pix - Image-to-Image Translation in PyTorch
T2T-ViT - ICCV2021, Tokens-to-Token ViT: Training Vision Transformers from Scratch on ImageNet
tf-keras-vis - Neural network visualization toolkit for tf.keras
multi-label-sentiment-classifier - How to build a multi-label sentiment classifiers with Tez and PyTorch
Transformer-MM-Explainability - [ICCV 2021- Oral] Official PyTorch implementation for Generic Attention-model Explainability for Interpreting Bi-Modal and Encoder-Decoder Transformers, a novel method to visualize any Transformer-based network. Including examples for DETR, VQA.
HugsVision - HugsVision is a easy to use huggingface wrapper for state-of-the-art computer vision
pytorch-tutorial - PyTorch Tutorial for Deep Learning Researchers
tf-metal-experiments - TensorFlow Metal Backend on Apple Silicon Experiments (just for fun)
Real-Time-Voice-Cloning - Clone a voice in 5 seconds to generate arbitrary speech in real-time
deep-text-recognition-benchmark - PyTorch code of my ICDAR 2021 paper Vision Transformer for Fast and Efficient Scene Text Recognition (ViTSTR)