VehicleFinder-CTIM
prismer
| VehicleFinder-CTIM | prismer | |
|---|---|---|
| 1 | 5 | |
| 5 | 1,305 | |
| - | 0.5% | |
| 3.1 | 5.2 | |
| over 1 year ago | about 1 year ago | |
| Python | Python | |
| - | GNU General Public License v3.0 or later |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
VehicleFinder-CTIM
-
FindVehicle and VehicleFinder: A NER dataset for natural language-based vehicle retrieval and a keyword-based cross-modal vehicle retrieval system
Natural language (NL) based vehicle retrieval is a task aiming to retrieve a vehicle that is most consistent with a given NL query from among all candidate vehicles. Because NL query can be easily obtained, such a task has a promising prospect in building an interactive intelligent traffic system (ITS). Current solutions mainly focus on extracting both text and image features and mapping them to the same latent space to compare the similarity. However, existing methods usually use dependency analysis or semantic role-labelling techniques to find keywords related to vehicle attributes. These techniques may require a lot of pre-processing and post-processing work, and also suffer from extracting the wrong keyword when the NL query is complex. To tackle these problems and simplify, we borrow the idea from named entity recognition (NER) and construct FindVehicle, a NER dataset in the traffic domain. It has 42.3k labelled NL descriptions of vehicle tracks, containing information such as the location, orientation, type and colour of the vehicle. FindVehicle also adopts both overlapping entities and fine-grained entities to meet further requirements. To verify its effectiveness, we propose a baseline NL-based vehicle retrieval model called VehicleFinder. Our experiment shows that by using text encoders pre-trained by FindVehicle, VehicleFinder achieves 87.7\% precision and 89.4\% recall when retrieving a target vehicle by text command on our homemade dataset based on UA-DETRAC. The time cost of VehicleFinder is 279.35 ms on one ARM v8.2 CPU and 93.72 ms on one RTX A4000 GPU, which is much faster than the Transformer-based system. The dataset is open-source via the link https://github.com/GuanRunwei/FindVehicle, and the implementation can be found via the link https://github.com/GuanRunwei/VehicleFinder-CTIM.
prismer
-
[D] Tracking Dancing People
tracking with An Ensemble of Experts similar to this https://github.com/NVlabs/Prismer
-
Meet Prismer: An Open Source Vision-Language Model with An Ensemble of Experts
Quick Read: https://www.marktechpost.com/2023/03/11/meet-prismer-an-open-source-vision-language-model-with-an-ensemble-of-experts/ Paper: https://arxiv.org/pdf/2303.02506.pdf Code: https://github.com/nvlabs/prismer
- Prismer: A Vision-Language Model with Multi-Modal Experts
-
[R] Prismer: An Open Source Vision-Language Model with An Ensemble of Experts.
Code and Models - https://github.com/NVlabs/prismer
What are some alternatives?
x-clip - A concise but complete implementation of CLIP with various experimental improvements from recent papers
CLIP-Caption-Reward - PyTorch code for "Fine-grained Image Captioning with CLIP Reward" (Findings of NAACL 2022)
FindVehicle - FindVehicle: A NER dataset in transportation to extract keywords describing vehicles on the road
Oscar - Oscar and VinVL
Macaw-LLM - Macaw-LLM: Multi-Modal Language Modeling with Image, Video, Audio, and Text Integration
InternGPT - InternGPT (iGPT) is an open source demo platform where you can easily showcase your AI models. Now it supports DragGAN, ChatGPT, ImageBind, multimodal chat like GPT-4, SAM, interactive image editing, etc. Try it at igpt.opengvlab.com (支持DragGAN、ChatGPT、ImageBind、SAM的在线Demo系统)
transformers - 🤗 Transformers: State-of-the-art Machine Learning for Pytorch, TensorFlow, and JAX.
Qwen-VL - The official repo of Qwen-VL (通义千问-VL) chat & pretrained large vision language model proposed by Alibaba Cloud.
clip-as-service - 🏄 Scalable embedding, reasoning, ranking for images and sentences with CLIP
open_clip - An open source implementation of CLIP.