Detic VS YOLOX

Some of the foundation/base models include: * GroundedSAM (Segment Anything Model) * DETIC * GroundingDINO

For localisation at search time I ended up using OWL-ViT. This worked really well. I did not try Detic or CLIPseg but would be interested to hear if anyone else has tried these?

git clone https://github.com/facebookresearch/Detic cd Detic pip install -r requirements python demo.py --config-file configs/Detic_LCOCOI21k_CLIP_SwinB_896b32_4x_ft4x_max-size.yaml --input desk.jpg --output out.jpg --vocabulary lvis --opts MODEL.WEIGHTS models/Detic_LCOCOI21k_CLIP_SwinB_896b32_4x_ft4x_max-size.pth

You might want to take a look at DETIC : https://github.com/facebookresearch/Detic (Open Vocabulary Object Detection, trained on thousands of classes)

Are there any open-label segmentation model included in this repo, like Detic or LSeg?

We made this using pretty recent advances in web-data pretrained models like Detic and LSeg for detection, CLIP for visual queries, and Sentence BERT for semantic queries. Our "database" is really a neural field (Instant NGP) that maps from 3D coordinates to a high dimensional embedding vector in the same representation space as CLIP and SBERT.

Is it similar to the open vocabulary detic?

Code for https://arxiv.org/abs/2201.02605 found: https://github.com/facebookresearch/Detic

github: https://github.com/facebookresearch/Detic

So I am trying to do my best and train YOLOX for an object detection case using Google Colab.

https://github.com/Megvii-BaseDetection/YOLOX I have only tried the pretrained yolo X nano. I get corner responses even if the inference image is padded with a large margin which is unexpected

* Nice implemention of Yolo that is BSD license (not GPL) https://github.com/Megvii-BaseDetection/YOLOX

I wanted to have a few choices getting localization into image search (index and search time). I immediately thought of using a region proposal network (rpn) from mask-rcnn to create patches that can also be indexed and searched (and add the localisation). I figured it might be somewhat agnostic to classes. I did not want to use mmdetection or detectron2 due to their dependencies and just getting the rpn was not worth it. I was encouraged by the PyTorch native implementations of detection/segmentation models but ended up finding yolox the best.

Megvii-BaseDetection/YOLOX

Modern video can be broken into a series of still problems. AI vision models can make these types of classification in as fast as video. Here is a particularly there is a controversial company from China that does this very well on faces in video and they have open sourced the models: https://github.com/Megvii-BaseDetection/YOLOX

Not really a problem, see results here. Just use yolox_x. Thank you for your attention.

From what I understand the swin transformer outputs a single dimension feature vector and the yolo head takes inputs from 3 different layers from the backbone?? and I think I will need to write the backbone implementation here.