clipseg
This repository contains the code of the CVPR 2022 paper "Image Segmentation Using Text and Image Prompts". (by timojl)
dino
PyTorch code for Vision Transformers training with the Self-Supervised learning method DINO (by facebookresearch)
| clipseg | dino | |
|---|---|---|
| 7 | 7 | |
| 1,022 | 5,904 | |
| - | 2.3% | |
| 3.8 | 1.0 | |
| 4 months ago | 3 days ago | |
| Python | Python | |
| GNU General Public License v3.0 or later | Apache License 2.0 |
The number of mentions indicates the total number of mentions that we've tracked plus the number of user suggested alternatives.
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
clipseg
Posts with mentions or reviews of clipseg.
We have used some of these posts to build our list of alternatives
and similar projects. The last one was on 2023-12-08.
-
How to blend a logo or clip art to a design
Following the comments to this old post, I tried to use in-painting with manual mask selection. I didn't get beautiful results but I'm sure with some tweaking, I could make it better. The main problem I had was having to manually select the area where I wanted to place the logo and trying to resize my logo mask to the fit the segment. I tried some automatic segmentation tools (Clipseg and Segment Anything). I couldn't tell the segmentation models to find a good area to for logo placement (i.e. some small flat surface). Given the complexity of what I was dealing with, I think there could be a better way (XY problem).
-
New Feature: "ZOOM ENHANCE" for the A111 WebUI. Automatically fix small details like faces and hands!
The addon utilizes clipseg for region masking, which was trained on "an extended version of the PhraseCut dataset" (many thousands of images.)
-
Txt2mask just received a big update!! 🎅
You'll also need to make sure to update your clipseg repo. The script won't do this for you. Namely you just need to update this models/clipseg.py file to ensure your clipseg has support for the new model.
-
[P] Image search with localization and open-vocabulary reranking.
For localisation at search time I ended up using OWL-ViT. This worked really well. I did not try Detic or CLIPseg but would be interested to hear if anyone else has tried these?
-
Who needs prompt2prompt anyway? SD 1.5 inpainting model with clipseg prompt for "hair" and various prompts for different hair colors
clipseg is an image segmentation method used to find a mask for an image from a prompt. I implemented it as an executor for dalle-flow and added it to my bot yasd-discord-bot.
-
txt2mask working in imaginAIry python library
Automated Replacement (txt2mask) by clipseg
- txt2mask was just released! We don't have to use the brush tool for inpainting anymore!
dino
Posts with mentions or reviews of dino.
We have used some of these posts to build our list of alternatives
and similar projects. The last one was on 2022-12-15.
- Batch-wise processing or image-by-image processing? (DINO V1)
-
[P] Image search with localization and open-vocabulary reranking.
I also implemented one based on the self attention maps from the DINO trained ViT’s. This worked pretty well when the attention maps were combined with some traditional computer vision to get bounding boxes. It seemed an ok compromise between domain specialization and location specificity. I did not try any saliency or gradient based methods as i was not sure on generalization and speed respectively. I know LAVIS has an implementation of grad cam and it seems to work well in the plug'n'play vqa.
-
Unsupervised semantic segmentation
You will probably need an unwieldy amount of data and compute to reproduce it, so your best option would be to use the pretrained models available on github.
-
[D] Why Transformers are taking over the Compute Vision world: Self-Supervised Vision Transformers with DINO explained in 7 minutes!
[Full Explanation Post] [Arxiv] [Project Page]
-
A major part of real-world AI has to be solved to make unsupervised, generalized full self-driving work, as the entire road system is designed for biological neural nets with optical imagers
Except he is actually talking about the new DINO model created by facebook that was released on friday. Which is a new approach to image transformers for unsupervised segmentation. Here's its github.
-
[D] Paper Explained - DINO: Emerging Properties in Self-Supervised Vision Transformers (Full Video Analysis)
Code: https://github.com/facebookresearch/dino
- [R] DINO and PAWS: Advancing the state of the art in computer vision with self-supervised Transformers
What are some alternatives?
When comparing clipseg and dino you can also consider the following projects:
stable-diffusion - Latent Text-to-Image Diffusion
simsiam-cifar10 - Code to train the SimSiam model on cifar10 using PyTorch
Detic - Code release for "Detecting Twenty-thousand Classes using Image-level Supervision".
Transformer-SSL - This is an official implementation for "Self-Supervised Learning with Swin Transformers".
LAVIS - LAVIS - A One-stop Library for Language-Vision Intelligence
pytorch-metric-learning - The easiest way to use deep metric learning in your application. Modular, flexible, and extensible. Written in PyTorch.
imaginAIry - Pythonic AI generation of images and videos
pytorch-lightning - Build high-performance AI models with PyTorch Lightning (organized PyTorch). Deploy models with Lightning Apps (organized Python to build end-to-end ML systems). [Moved to: https://github.com/Lightning-AI/lightning]
txt2mask - Automatically create masks for Stable Diffusion inpainting using natural language.
unsupervised-depth-completion-visual-inertial-odometry - Tensorflow and PyTorch implementation of Unsupervised Depth Completion from Visual Inertial Odometry (in RA-L January 2020 & ICRA 2020)
dalle-flow - 🌊 A Human-in-the-Loop workflow for creating HD images from text
lightly - A python library for self-supervised learning on images.
clipseg vs stable-diffusion
dino vs simsiam-cifar10
clipseg vs Detic
dino vs Transformer-SSL
clipseg vs LAVIS
dino vs pytorch-metric-learning
clipseg vs imaginAIry
dino vs pytorch-lightning
clipseg vs txt2mask
dino vs unsupervised-depth-completion-visual-inertial-odometry
clipseg vs dalle-flow
dino vs lightly