mmdeploy VS mmdetection

The RK3588 also has a NPU for accelerating neural networks. The bad news is the API is not supported by any of the inference engines (afaik), but the NPU can run models directly that have been converted to the RKNN format. It is a long shot, but you can find details here.

BibTeX @misc{=mmdeploy, title={OpenMMLab's Model Deployment Toolbox.}, author={MMDeploy Contributors}, howpublished = {\url{https://github.com/open-mmlab/mmdeploy}}, year={2021} }

I have a semantic segmentation model trained using the mmdetection repo. Then it is converted to the ONNX format using the mmdeploy repo.

There certainly are issues on Linux as well. The Detectron2 library alone has several hundred issues related to incorrect versions of something: https://github.com/facebookresearch/detectron2/issues

The mmdetection library (https://github.com/open-mmlab/mmdetection/issues) also has hundreds of version-related issues. Admittedly, that library has not seen any updates for over a year now, but it is sad that things just break and become basically unusable on modern Linux operating systems because NVIDIA can't stop breaking backwards and forwards compatibility for what is essentially just fancy matrix multiplication.

When I look for benchmarks I always start here https://paperswithcode.com/task/instance-segmentation/codeless it has the lists of datasets to measure models accross lots o papers. Many are very specific models with low support or community but it gives you a good idea of ​​the state of the art. It also lists repositories related to good community. https://github.com/open-mmlab/mmdetection seems very active and the one that is being used the most, you could use the models that it has integrated in its model zoo, within the same repository. It has the benchmarks to compare those same models and some of them are from 2022

MODEL: https://github.com/open-mmlab/mmdetection

Would it make sense to work with OpenMMLab (https://github.com/open-mmlab/mmdetection) or Pytorch-image-models (https://github.com/rwightman/pytorch-image-models#models) since they offer a variety of models?

I wanted to have a few choices getting localization into image search (index and search time). I immediately thought of using a region proposal network (rpn) from mask-rcnn to create patches that can also be indexed and searched (and add the localisation). I figured it might be somewhat agnostic to classes. I did not want to use mmdetection or detectron2 due to their dependencies and just getting the rpn was not worth it. I was encouraged by the PyTorch native implementations of detection/segmentation models but ended up finding yolox the best.

MMDetection: OpenMMLab detection toolbox and benchmark.

I have a semantic segmentation model trained using the mmdetection repo. Then it is converted to the ONNX format using the mmdeploy repo.

E.g. If you consider it an object detection problem it is: detect and localise all the pedestrians in a frame, and classify them by their (intended) action. IMO the easiest way to do this would be with mmdetection, which is built on top of pytorch. Just label your dataset, build a config, and boom you have a model. Inference with that model in only a few lines of code, you won't really need to learn too much to get started.

MMDetection

For example, in mmdetection, they expose options in their config & implementation to freeze batch norm layers in backbones and in this config, norm_eval is set to True meaning to freeze tracking of batch norm stats, while the ResNet backbone is frozen up to the 1st stage. Example of their backbone implementation can be found here.