The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning. Learn more →
Top 23 Dataset Open-Source Projects
-
-
InfluxDB
Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
-
label-studio
Label Studio is a multi-type data labeling and annotation tool with standardized output format
-
-
-
cleanlab
The standard data-centric AI package for data quality and machine learning with messy, real-world data and labels.
-
WorkOS
The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.
-
-
awesome-project-ideas
Curated list of Machine Learning, NLP, Vision, Recommender Systems Project Ideas
-
-
browser-compat-data
This repository contains compatibility data for Web technologies as displayed on MDN
-
esProc
esProc SPL is a scripting language for data processing, with well-designed rich library functions and powerful syntax, which can be executed in a Java program through JDBC interface and computing independently.
-
awesome-pretrained-chinese-nlp-models
Awesome Pretrained Chinese NLP Models，高质量中文预训练模型&大模型&多模态模型&大语言模型集合
-
-
sql-translator
SQL Translator is a tool for converting natural language queries into SQL code using artificial intelligence. This project is 100% free and open source.
-
-
img2dataset
Easily turn large sets of image urls to an image dataset. Can download, resize and package 100M urls in 20h on one machine.
-
covid-chestxray-dataset
We are building an open database of COVID-19 cases with chest X-ray or CT images.
-
-
whylogs
An open-source data logging library for machine learning models and data pipelines. 📚 Provides visibility into data quality & model performance over time. 🛡️ Supports privacy-preserving data collection, ensuring safety & robustness. 📈
-
SaaSHub
SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives
✅ public-apis/public-apis : https://github.com/public-apis/public-apis
Faker was originally written in Perl and is also available as a library for Ruby, Java, and Python.
14. LabelStudio by Human Signal | Github | tutorial
Project mention: Logistic Regression for Image Classification Using OpenCV | news.ycombinator.com | 2023-12-31In this case there's no advantage to using logistic regression on an image other than the novelty. Logistic regression is excellent for feature explainability, but you can't explain anything from an image.
Traditional classification algorithms but not deep learning such as SVMs and Random Forest perform a lot better on MNIST, up to 97% accuracy compared to the 88% from logistic regression in this post. Check the Original MNIST benchmarks here: http://fashion-mnist.s3-website.eu-central-1.amazonaws.com/#
Huh?
I wrote my own system for classifying a stream of texts in Python, I might Open Source it one of these days but I have to get it to the point where it is modular enough that I can customize it to do the particular things I want without subjecting people to my whims... I use it every day and I'm not afraid to demo it because it is rock solid.
My understanding is that my system would not be hard to adapt to work on images for certain kinds of tasks.
Pytorch is open source, Huggingface is open source. CUDA isn't. This is
https://labelstud.io/
and for annotating text spans there are so many open source tools
https://github.com/doccano/doccano
I worked for a company a few years back that built annotation tools for projects we sold to customers but never quite got to a polished general purpose annotator. Today there are an overwhelming number of companies in this space and products I never heard of, many of which are cloud based or paid. Looks like a gold rush to me.
Project mention: [Research] Detecting Annotation Errors in Semantic Segmentation Data | /r/MachineLearning | 2023-11-05We have feely open-sourced our new method for improving segmentation data, published a paper on the research behind it, and released a 5-min code tutorial. You can also read more in the blog if you'd like.
Project mention: Here are the 10 projects I am contributing to over the next 6 months. Share yours | dev.to | 2024-04-13
I started to see more and more applications that use the OpenAI API and I wanted to try it out. One of these apps is this one made by Kate.
Project mention: OpenAI sued for web scraping from millions of internet users in order to train ChatGPT | /r/ArtistHate | 2023-06-30Lmao, no it doesn't. As we can see, their downloader uses very obscure "no ai" headers (which can be disabled, so its useless). They only claim it respects "robots.txt" because the google crawler respects it, if a site changes their robots.txt rules they don't remove it from their dataset, that is not "respecting". https://github.com/rom1504/img2dataset
I've looked at https://github.com/pydata/pandas-datareader and it looks good, does anyone have experience?
NOTE:
The open source projects on this list are ordered by number of github stars.
The number of mentions indicates repo mentiontions in the last 12 Months or
since we started tracking (Dec 2020).
Dataset related posts
- Computing Engine on Web
- What is the difference between BI and AI？
- Multi Purpose Traversal
- Natural Earth – Free vector and raster map data
- Why ETL Becomes ELT or Even Let？
- A tool for developing quantitative strategy model
- An Open Source Tool for Multimodal Fact Verification
-
A note from our sponsor - WorkOS
workos.com | 25 Apr 2024
Index
What are some of the best open-source Dataset projects? This list will help you:
| Project | Stars | |
|---|---|---|
| 1 | public-apis | 292,037 |
| 2 | faker | 17,080 |
| 3 | label-studio | 16,469 |
| 4 | fashion-mnist | 11,439 |
| 5 | LaTeX-OCR | 10,711 |
| 6 | doccano | 8,966 |
| 7 | cleanlab | 8,592 |
| 8 | techniques | 7,739 |
| 9 | awesome-project-ideas | 7,404 |
| 10 | quickdraw-dataset | 5,934 |
| 11 | browser-compat-data | 4,777 |
| 12 | esProc | 4,425 |
| 13 | awesome-pretrained-chinese-nlp-models | 4,193 |
| 14 | datasets | 4,162 |
| 15 | sql-translator | 3,966 |
| 16 | Chinese-Names-Corpus | 3,814 |
| 17 | text | 3,441 |
| 18 | img2dataset | 3,242 |
| 19 | awesome-json-datasets | 3,183 |
| 20 | TextRecognitionDataGenerator | 3,038 |
| 21 | covid-chestxray-dataset | 2,958 |
| 22 | pandas-datareader | 2,819 |
| 23 | whylogs | 2,543 |
Sponsored
SaaSHub - Software Alternatives and Reviews
SaaSHub helps you find the best software and product alternatives
www.saashub.com