-
Mage
🧙 The modern replacement for Airflow. Mage is an open-source data pipeline tool for transforming and integrating data. https://github.com/mage-ai/mage-ai
-
InfluxDB
Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
If you want to try out building data pipelines on cloud resources, you can try out this tool: http://demo.mage.ai/. Here is the project: https://github.com/mage-ai/mage-ai
Also, I don’t know if you have come across this but there is a free data engineering course available from DataTalksClub going over some GCP features, Terraform, Docker, Spark, Airflow, and more. I found it great to get a feeling of how all these tools interact.