deep-text-recognition-benchmark
Transformer-Explainability
| deep-text-recognition-benchmark | Transformer-Explainability | |
|---|---|---|
| 1 | 1 | |
| 278 | 1,677 | |
| - | - | |
| 2.2 | 0.0 | |
| about 1 month ago | 4 months ago | |
| Jupyter Notebook | Jupyter Notebook | |
| Apache License 2.0 | MIT License |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
deep-text-recognition-benchmark
-
Building an Internet Scale Meme Search Engine
https://github.com/roatienza/deep-text-recognition-benchmark (available weights are for tasks that seem similar to OCR so there is a good chance you can use it out of the box). With a good gpu it should process hundreds to thousands image per seconds, so you likely can build your index in less than a day. (Maybe you can even port it to your iphone stack :) )
https://github.com/microsoft/GenerativeImage2Text (You'll probably have to train on your custom dataset that you have constituted)
There are tons of other freely available solutions that you can get with a search for things with keywords like "image to text ocr" "transformers" "visual transformers"...
Transformer-Explainability
-
[Project] Recent Class Activation Map Methods for CNNs and Vision Transformers
Not exactly the same but since you mentioned using ViT's attention outputs as a 2D feature map for the CAM you can consider this paper (Transformer Interpretability Beyond Attention Visualization) where they study the question of how to choose/mix the attention scores in a way that can be visualized (so similar to the CAMs). Maybe it can lead to better results. https://arxiv.org/abs/2012.09838 https://github.com/hila-chefer/Transformer-Explainability
What are some alternatives?
GenerativeImage2Text - GIT: A Generative Image-to-text Transformer for Vision and Language
pytorch-grad-cam - Advanced AI Explainability for computer vision. Support for CNNs, Vision Transformers, Classification, Object detection, Segmentation, Image similarity and more.
Calliar - A dataset for online Arabic calligraphy. A collection of 2500 annotated calligraphic styles.
shap - A game theoretic approach to explain the output of any machine learning model.
sonic - 🦔 Fast, lightweight & schema-less search backend. An alternative to Elasticsearch that runs on a few MBs of RAM.
T2T-ViT - ICCV2021, Tokens-to-Token ViT: Training Vision Transformers from Scratch on ImageNet
ocrit - Simple command-line utility for performing OCR using Apple's Vision framework
multi-label-sentiment-classifier - How to build a multi-label sentiment classifiers with Tez and PyTorch
macOCR - Get any text on your screen into your clipboard.
HugsVision - HugsVision is a easy to use huggingface wrapper for state-of-the-art computer vision
Transformers-Tutorials - This repository contains demos I made with the Transformers library by HuggingFace.
tf-metal-experiments - TensorFlow Metal Backend on Apple Silicon Experiments (just for fun)