Data as a build system ?

This page summarizes the projects mentioned and recommended in the original post on /r/dataengineering
Data Science Machine Learning bioinformatics-pipeline Workflow Scientific Computing

InfluxDB - Power Real-Time Data Analytics at Scale
Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
www.influxdata.com
featured
SaaSHub - Software Alternatives and Reviews
SaaSHub helps you find the best software and product alternatives
www.saashub.com
featured

  • dvc

    🦉 ML Experiments and Data Management with Git

    • This idea came from https://dvc.org/ which is aimed at ML and versioning models, but I think they definitely got a point.

  • reflow

    A language and runtime for distributed, incremental data processing in the cloud

    • https://github.com/grailbio/reflow is the closest that I know, as it has a design that resembles the Bazel build system.

  • InfluxDB

    Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

    InfluxDB logo

  • projects

    Sample projects using Ploomber. (by ploomber)

    • SQL pipeline

  • ploomber

    The fastest ⚡️ way to build data pipelines. Develop iteratively, deploy anywhere. ☁️

    • Github

NOTE: The number of mentions on this list indicates mentions on common posts plus user suggested alternatives. Hence, a higher number means a more popular project.

Suggest a related project

Related posts

  • Show HN: JupySQL – a SQL client for Jupyter (ipython-SQL successor)

    2 projects | news.ycombinator.com | 6 Dec 2023

  • New to large SW projects in Python, best practices to organize code

    1 project | /r/Python | 11 Nov 2022

  • A three-part series on deploying a Data Science Platform on AWS

    1 project | /r/dataengineering | 4 Nov 2022

  • Ploomber Cloud - Parametrizing and running notebooks in the cloud in parallel

    3 projects | /r/IPython | 3 Nov 2022

  • Is Colab still the place to go?

    1 project | /r/deeplearning | 2 Nov 2022