What are some good publicly available real-time data sources?

This page summarizes the projects mentioned and recommended in the original post on /r/dataengineering

Our great sponsors
  • SurveyJS - Open-Source JSON Form Builder to Create Dynamic Forms Right in Your App
  • WorkOS - The modern identity platform for B2B SaaS
  • InfluxDB - Power Real-Time Data Analytics at Scale
  • awesome-public-real-time-datasets

    A list of publicly available datasets with real-time data maintained by the team at bytewax.io

  • I am attempting to source via the wisdom of the crowd here. I often find it hard to find good real-time data sources for learning about streaming, prototyping, or building hobby projects. I started researching and then created an "Awesome List" in a GitHub repo - https://github.com/bytewax/awesome-public-real-time-datasets.

  • RedfinScraper

    Scrapes Redfin data.

  • Shameless plug for RedfinScraper

  • SurveyJS

    Open-Source JSON Form Builder to Create Dynamic Forms Right in Your App. With SurveyJS form UI libraries, you can build and style forms in a fully-integrated drag & drop form builder, render them in your JS app, and store form submission data in any backend, inc. PHP, ASP.NET Core, and Node.js.

    SurveyJS logo
  • eventsim

    Event data simulator. Generates a stream of pseudo-random events from a set of users, designed to simulate web traffic. (by viirya)

  • In my last real-time sideproject, I used an open source music app simulator called Eventsim. The original doesn't work anymore, but this repo still did when I used it around Nov/Dec.

  • mockingbird

    Mockingbird is a mock streaming data generator (by tinybirdco)

  • owl-shop

  • There is owl shop https://github.com/cloudhut/owl-shop

  • datagen

    Generate authentic looking mock data based on a SQL, JSON or Avro schema and produce to Kafka in JSON or Avro format.

NOTE: The number of mentions on this list indicates mentions on common posts plus user suggested alternatives. Hence, a higher number means a more popular project.

Suggest a related project

Related posts