Show HN: Simple (but clever) algorithms can find label issues in datasets

• vizzy

  3 12 2.8 TypeScript

  Cleanlab Vizzy: illustrating the core ideas behind the Cleanlab algorithm (by cleanlab)

  

• cleanlab

  69 8,651 9.4 Python

  The standard data-centric AI package for data quality and machine learning with messy, real-world data and labels.

  

I built Vizzy as a hackathon project, with the goal of explaining how relatively simple (but clever) algorithms can be a powerful tool to automatically find issues in datasets, including label errors and out-of-distribution data.

Vizzy uses a JavaScript port of (a part of) https://github.com/cleanlab/cleanlab, which implements the algorithms described in https://arxiv.org/abs/1911.00068.

There are other neat technical nuggets in the implementation of Vizzy as well, including ML model training in the browser (using features from a pretrained ResNet-18, performing truncated SVD, and using an SVM model for speed). If you’re interested in the details of how Vizzy works, check out this blog post: https://cleanlab.ai/blog/cleanlab-vizzy/

I’m happy to answer any questions related to Vizzy, cleanlab, or confident learning and data-centric AI in general!

• InfluxDB

  www.influxdata.com sponsored

  Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

  

Suggest a related project