anomalib VS examples

This talk highlights the role of Anomalib, an open-source deep learning framework, in advancing anomaly detection within AI systems, particularly showcased at the upcoming CVPR Visual Anomaly and Novelty Detection (VAND) workshop. Anomalib integrates advanced algorithms and tools to facilitate both academic research and practical applications in sectors like manufacturing, healthcare, and security. It features capabilities such as experiment tracking, model optimization, and scalable deployment solutions. Additionally, the discussion will include Anomalib’s participation in the VAND challenge, focusing on robust real-world applications and few-shot learning for anomaly detection.

Then, when it comes to semi-supervised learning for anomaly detection, I had positive experiences with Anomalib which offers a robust library dedicated to deep learning anomaly detection algorithms. They implemented the latest models with PyTorch and offer tools to benchmark their performance.

Anomalib is an open-source library for unsupervised anomaly detection in images. It offers a collection of state-of-the-art models that can be trained on your specific images.

Anomalib is an open-source deep learning library developed by Intel that makes it easy to benchmark different anomaly detection algorithms on both public and custom datasets, all by simply modifying a config file. As the largest public collection of anomaly detection algorithms and datasets, it has a strong focus on image-based anomaly detection. It’s a comprehensive, end-to-end solution that includes cutting-edge algorithms, relevant evaluation methods, prediction visualizations, hyperparameter optimization, and inference deployment code with Intel’s OpenVINO Toolkit.

try https://github.com/openvinotoolkit/anomalib it's primarily aimed at vision applications but might provide some inspiration

Anomaly detection typically refers to the task of finding unusual or rare items that deviate significantly from what is considered to be the "normal" majority. In this blogpost, we look at image anomalies using PatchCore. Next to indicating which images are anomalous, PatchCore also identifies the most anomalous pixel regions within each image. One big advantage of PatchCore is that it only requires normal images for training, making it attractive for many use cases where abnormal images are rare or expensive to acquire. In some cases, we don't even know all the unusual patterns that we might encounter and training a supervised model is not an option. One example use case is the detection of defects in industrial manufacturing, where most defects are rare by definition as production lines are optimised to produce as few of them as possible. Recent approaches have made significant progress on anomaly detection in images, as demonstrated on the MVTec industrial benchmark dataset. PatchCore, presented at CVPR 2022, is one of the frontrunners in this field. In this blog post we first dive into the inner workings of PatchCore. Next, we apply it to an example in medical imaging to gauge its applicability outside of industrial examples. We use the anomalib library, which was developed by Intel and offers ready-to-use implementations of many recent image anomaly detection methods.

As another commenter noted, Milvus is overkill and a "bit much" if you're learning/playing.

A good intro to the field with progression towards a full Milvus implementation could be starting with towhee[0] (which is also supported by Milvus).

towhee has an example to do exactly what you want with CLIP[1].

[0] - https://towhee.io/

[1] - https://github.com/towhee-io/examples/tree/main/image/text_i...

Usually this is done in three steps. The first step is using a neural network to create a bounding box around the object, then generating vector embeddings of the object, and then using similarity search on vector embeddings.

The first step is accomplished by training a detection model to generate the bounding box around your object, this can usually be done by finetuning an already trained detection model. For this step the data you would need is all the images of the object you have with a bounding box created around it, the version of the object doesnt matter here.

The second step involves using a generalized image classification model thats been pretrained on generalized data (VGG, etc.) and a vector search engine/vector database. You would start by using the image classification model to generate vector embeddings (https://frankzliu.com/blog/understanding-neural-network-embe...) of all the different versions of the object. The more ground truth images you have, the better, but it doesn't require the same amount as training a classifier model. Once you have your versions of the object as embeddings, you would store them in a vector database (for example Milvus: https://github.com/milvus-io/milvus).

Now whenever you want to detect the object in an image you can run the image through the detection model to find the object in the image, then run the sliced out image of the object through the vector embedding model. With this vector embedding you can then perform a search in the vector database, and the closest results will most likely be the version of the object.

Hopefully this helps with the general rundown of how it would look like. Here is an example using Milvus and Towhee https://github.com/towhee-io/examples/tree/3a2207d67b10a246f....

Disclaimer: I am a part of those two open source projects.

I have shown how to Build an Image Search Engine in Minutes in the previous tutorial. Here is another one for how to optimize the algorithm, feed it with large-scale image datasets, and deploy it as a micro-service.

The full tutorial is at https://github.com/towhee-io/examples/blob/main/image/reverse_image_search/build_image_search_engine.ipynb