Efficient-AI-Backbones
PyTorch-Vision-Transformer-ViT-MNIST-CIFAR10
| Efficient-AI-Backbones | PyTorch-Vision-Transformer-ViT-MNIST-CIFAR10 | |
|---|---|---|
| 3 | 2 | |
| 3,816 | 60 | |
| 1.5% | - | |
| 5.8 | 8.8 | |
| 6 days ago | 3 days ago | |
| Python | Python | |
| - | - |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
Efficient-AI-Backbones
-
Researchers From China Introduce Vision GNN (ViG): A Graph Neural Network For Computer Vision Systems
Continue reading | Check out the paper, github
- GNN for computer vision, beating CNN & Transformer
-
GNN can also work well on computer vision
Vision GNN: An Image is Worth Graph of Nodes Network architecture plays a key role in the deep learning-based computer vision system. The widely-used convolutional neural network and transformer treat the image as a grid or sequence structure, which is not flexible to capture irregular and complex objects. In this paper, we propose to represent the image as a graph structure and introduce a new Vision GNN (ViG) architecture to extract graph-level feature for visual tasks. We first split the image to a number of patches which are viewed as nodes, and construct a graph by connecting the nearest neighbors. Based on the graph representation of images, we build our ViG model to transform and exchange information among all the nodes. ViG consists of two basic modules: Grapher module with graph convolution for aggregating and updating graph information, and FFN module with two linear layers for node feature transformation. Both isotropic and pyramid architectures of ViG are built with different model sizes. Extensive experiments on image recognition and object detection tasks demonstrate the superiority of our ViG architecture. We hope this pioneering study of GNN on general visual tasks will provide useful inspiration and experience for future research. The PyTroch code will be available at https://github.com/huawei-noah/CV-Backbones.
PyTorch-Vision-Transformer-ViT-MNIST-CIFAR10
-
Scratch Implementation of Vision Transformer in PyTorch
In the encoder class, ViTs use prenorm not post-norm like BERT. That is the first norm layer should be before the attention and the second norm layer should be before "self.fc1" https://github.com/s-chh/PyTorch-Vision-Transformer-ViT-MNIST/blob/main/model.py
What are some alternatives?
MPViT - [CVPR 2022] MPViT:Multi-Path Vision Transformer for Dense Prediction
SwinIR - SwinIR: Image Restoration Using Swin Transformer (official repository)
FQ-ViT - [IJCAI 2022] FQ-ViT: Post-Training Quantization for Fully Quantized Vision Transformer
towhee - Towhee is a framework that is dedicated to making neural data processing pipelines simple and fast.
transfiner - Mask Transfiner for High-Quality Instance Segmentation, CVPR 2022
PaddleViT - :robot: PaddleViT: State-of-the-art Visual Transformer and MLP Models for PaddlePaddle 2.0+
RethinkVSRAlignment - (NIPS 2022) Rethinking Alignment in Video Super-Resolution Transformers
mmdetection - OpenMMLab Detection Toolbox and Benchmark
deepvision - PyTorch and TensorFlow/Keras image models with automatic weight conversions and equal API/implementations - Vision Transformer (ViT), ResNetV2, EfficientNetV2, NeRF, SegFormer, MixTransformer, (planned...) DeepLabV3+, ConvNeXtV2, YOLO, etc.
LaTeX-OCR - pix2tex: Using a ViT to convert images of equations into LaTeX code.
Pretrained-Language-Model - Pretrained language model and its related optimization techniques developed by Huawei Noah's Ark Lab.
EfficientFormer - EfficientFormerV2 [ICCV 2023] & EfficientFormer [NeurIPs 2022]