efficientdet-pytorch VS mmsegmentation

However, there are papers on oriented object detectors (see https://arxiv.org/pdf/1911.07732.pdf) for example. In that paper, they do achieve better results using oriented bounding boxes. If you want to go down that route, I would suggest using the EfficientDet model, because the PyTorch code that you'll find for it is quite easy to understand and modify. For example, I've taken https://github.com/zylo117/Yet-Another-EfficientDet-Pytorch, and modified it to include a "thing-ness" logit, and this was pretty easy to do. Classic EfficientDet models only include logits (aka output neurons that get softmax-ed) for each class, and if any one of these class neurons is greater than 0.5, then it is considered "a thing". Anyway - that's digression, but my point is that I've thought about adding oriented box support to an EfficientDet model, and it didn't seem to be too hard, although I haven't actually done it. If I was to start now, I would probably go with https://github.com/rwightman/efficientdet-pytorch, since Ross Wightman's models are becoming a de-facto standard in the PyTorch world for all things image-related.

Segmentation models trained on datasets like ADE20k could probably be used for that, because it has separate classes for these things iirc. https://github.com/open-mmlab/mmsegmentation should have suitable pretrained models available.

MMSegmentation: OpenMMLab semantic segmentation toolbox and benchmark.

This repo is amazing: https://github.com/open-mmlab/mmsegmentation

Mmsegmentation would be a good place to start for basic segmentation. They have lots of recent methods and pretained models you could fine-tune from. They also support quite a few datasets including VOC. There is a custom dataset format which looks straightforward to create.