Roboflow 100: A New Object Detection Benchmark

This page summarizes the projects mentioned and recommended in the original post on news.ycombinator.com

Scout Monitoring - Free Django app performance insights with Scout Monitoring
Get Scout setup in minutes, and let us sweat the small stuff. A couple lines in settings.py is all you need to start monitoring your apps. Sign up for our free tier today.
www.scoutapm.com
featured
InfluxDB - Power Real-Time Data Analytics at Scale
Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
www.influxdata.com
featured
  • roboflow-100-benchmark

    Code for replicating Roboflow 100 benchmark results and programmatically downloading benchmark datasets

  • Scout Monitoring

    Free Django app performance insights with Scout Monitoring. Get Scout setup in minutes, and let us sweat the small stuff. A couple lines in settings.py is all you need to start monitoring your apps. Sign up for our free tier today.

    Scout Monitoring logo
  • roboflow-100-benchmark

    Discontinued Code for replicating Roboflow 100 benchmark results and programmatically downloading benchmark datasets [Moved to: https://github.com/roboflow/roboflow-100-benchmark] (by roboflow-ai)

    Thanks for sharing @jonbaer! I’m one of the co-founders of Roboflow. Some additional resources and context:

    * Blog Post: https://blog.roboflow.com/roboflow-100/

    * Paper: https://arxiv.org/abs/2211.13523

    * Github: https://github.com/roboflow-ai/roboflow-100-benchmark

    At Roboflow, we've seen users fine-tune hundreds of thousands of computer vision models on custom datasets.

    We observed that there's a huge disconnect between the types of tasks people are actually trying to perform in the wild and the types of datasets researchers are benchmarking their models on.

    Datasets like MS COCO (with hundreds of thousands of images of common objects) are often used in research to compare models' performance, but then those models are used to find galaxies, look at microscope images, or detect manufacturing defects in the wild (often trained on small datasets containing only a few hundred examples). This leads to big discrepancies in models' stated and real-world performance.

  • sahi

    Framework agnostic sliced/tiled inference + interactive ui + error analysis plots

    Good idea. I haven’t looked too closely yet at the “hard” datasets.

    We originally considered “fixing” the labels on these datasets by hand, but ultimately decided that label error is one of the challenges “real world” datasets have that models should work to become more robust against. There is some selection bias in that we did make sure that the datasets we chose passed the eye test (in other words, it looked like the user spent a considerable amount of time annotating & a sample of the images looked like they labeled some object of interest).

    For aerial images in particular my guess would be that these models suffer from the “small object problem”[1] where the subjects are tiny compared to the size of the image. Trying a sliding window based approach like SAHI[2] on them would probably produce much better results (at the expense of much lower inference speed).

    [1] https://blog.roboflow.com/detect-small-objects/

    [2] https://github.com/obss/sahi

  • make-sense

    Free to use online tool for labelling photos. https://makesense.ai

    Haven't heard of those two, but would be really awesome to see an integration. We have an open API[1] for just this reason: we really want to make it easy to use (and source) your data across all the different tools out there. We've recently launched integrations with other labeling[2] and AutoML[3] tools (and have integrations with the big-cloud AutoML tools as well[4]). We're hoping to have a bunch more integrations with other MLOps tools & platforms in 2023.

    Re synthetic data specifically, we've written a couple of how-to guides for creating data from context augmentation[5], Unity Perception[6], and Stable Diffusion[7] & are talking to some others as well; it seems like a natural integration point (and someplace where we don't need to reinvent the wheel).

    [1] https://docs.roboflow.com/rest-api

    [2] https://github.com/SkalskiP/make-sense/pull/298

    [3] https://github.com/ultralytics/yolov5/discussions/10425

    [4] https://docs.roboflow.com/train/pro-third-party-training-int...

    [5] https://blog.roboflow.com/how-to-create-a-synthetic-dataset-...

    [6] https://blog.roboflow.com/unity-perception-synthetic-dataset...

    [7] https://blog.roboflow.com/synthetic-data-with-stable-diffusi...

  • yolov5

    YOLOv5 🚀 in PyTorch > ONNX > CoreML > TFLite

    Haven't heard of those two, but would be really awesome to see an integration. We have an open API[1] for just this reason: we really want to make it easy to use (and source) your data across all the different tools out there. We've recently launched integrations with other labeling[2] and AutoML[3] tools (and have integrations with the big-cloud AutoML tools as well[4]). We're hoping to have a bunch more integrations with other MLOps tools & platforms in 2023.

    Re synthetic data specifically, we've written a couple of how-to guides for creating data from context augmentation[5], Unity Perception[6], and Stable Diffusion[7] & are talking to some others as well; it seems like a natural integration point (and someplace where we don't need to reinvent the wheel).

    [1] https://docs.roboflow.com/rest-api

    [2] https://github.com/SkalskiP/make-sense/pull/298

    [3] https://github.com/ultralytics/yolov5/discussions/10425

    [4] https://docs.roboflow.com/train/pro-third-party-training-int...

    [5] https://blog.roboflow.com/how-to-create-a-synthetic-dataset-...

    [6] https://blog.roboflow.com/unity-perception-synthetic-dataset...

    [7] https://blog.roboflow.com/synthetic-data-with-stable-diffusi...

  • InfluxDB

    Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

    InfluxDB logo
NOTE: The number of mentions on this list indicates mentions on common posts plus user suggested alternatives. Hence, a higher number means a more popular project.

Suggest a related project

Related posts

  • AI That Teaches Other AI

    4 projects | news.ycombinator.com | 20 Jul 2023
  • Show HN: I am using AI to drop hats outside my window onto New Yorkers

    6 projects | news.ycombinator.com | 23 Jun 2024
  • A History of CLIP Model Training Data Advances

    8 projects | dev.to | 13 Mar 2024
  • How to Build a Semantic Search Engine for Emojis

    6 projects | dev.to | 10 Jan 2024
  • Show HN: Lightly – A Python library for self-supervised learning on images

    1 project | news.ycombinator.com | 16 Nov 2023