-
weather_data_pipeline
This is a PySpark-based data pipeline that fetches weather data for a few cities, performs some basic processing and transformation on the data, and then writes the processed data to a Google Cloud Storage bucket and a BigQuery table.The data is then viewed in a looker dashboard
-
InfluxDB
Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
We'll be using PySpark for distributed data processing, Prefect for workflow management, and Google Cloud Storage and BigQuery for data storage and processing.The code is available on github.
NOTE:
The number of mentions on this list indicates mentions on common posts plus user suggested alternatives.
Hence, a higher number means a more popular project.
Related posts
-
Show HN: PgQueuer – Over 5k Jobs/SEC with PostgreSQL
-
Introducing Tapyr: Create and Deploy Enterprise-Ready PyShiny Dashboards with Ease
-
Sequoia: Serving exact Llama2-70B on an RTX4090 with 1/2 s per token
-
Github Sponsor Sebastián Ramírez Python programmer
-
Ask HN: Have you coded any productivity software just for yourself?