Building a pipeline in GCP

• ClashOfClans_API

  2 10 1.8 Python

  This utilizes a static IP address in GCP to pull data from the Clash of Clans API, store it and serve it via data studio. There is also a API client with a sleep of functions you can use to pull, store, and view CoC facts and dimensions.

First picture shows this architecture. (You won't need the VPC or NAT in your case): https://github.com/jaredfiacco2/ClashOfClans_API

• Healthchecks

  208 7,369 9.7 Python

  Open-source cron job and background task monitoring service, written in Python & Django

  

Not sure if you have thought about monitoring, but I like to use https://healthchecks.io/ to monitor when the process starts and ends.

• InfluxDB

  www.influxdata.com featured

  Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

  

Suggest a related project