Data Load Diagram

This page summarizes the projects mentioned and recommended in the original post on /r/dataengineering

Our great sponsors
  • InfluxDB - Access the most powerful time series database as a service
  • Sonar - Write Clean Python Code. Always.
  • SaaSHub - Software Alternatives and Reviews
  • grai-core

    We've been looking at building something like this for Grai specifically to support Airflow but haven't yet prioritized it.

  • sqlparse

    A non-validating SQL parser module for Python

    Gotcha, since we haven't actually written all of this yet I don't have any useful code snippets to share but we've discussed tackling the problem internally using something like sqlparse. You'd need to identify the relevant sql chunks, parse them for table dependency information and then create the relevant entities in whichever data lineage tool you were using.

  • InfluxDB

    Access the most powerful time series database as a service. Ingest, store, & analyze all types of time series data in a fully-managed, purpose-built database. Keep data forever with low-cost storage and superior data compression.

NOTE: The number of mentions on this list indicates mentions on common posts plus user suggested alternatives. Hence, a higher number means a more popular project.

Suggest a related project

Related posts