weather_data_pipeline

This is a PySpark-based data pipeline that fetches weather data for a few cities, performs some basic processing and transformation on the data, and then writes the processed data to a Google Cloud Storage bucket and a BigQuery table.The data is then viewed in a looker dashboard (by 24jmwangi)

Weather_data_pipeline Alternatives

Similar projects and alternatives to weather_data_pipeline

  1. prefect-deployment-patterns

    Code examples showing flow deployment to various types of infrastructure

  2. Kargo

    Stop Scripting Promotions. Start Shipping with Kargo. Kargo automates promotion across dev, staging, and prod with approval gates and verification. Open source, built by the team behind Argo CD. Download now.

    Kargo logo
  3. f1-data-pipeline

    Discontinued F1 Data Pipeline [GET https://api.github.com/repos/InosRahul/f1-data-pipeline: 404 - Not Found // See: https://docs.github.com/rest/repos/repos#get-a-repository]

  4. dataproc-templates

    Discontinued Dataproc templates and pipelines for solving in-cloud data tasks

  5. magic-the-gathering

    A complete pipeline to pull data from Scryfall's "Magic: The Gathering"-API, via Prefect orchestration and dbt transformation.

  6. maternal-health-risk

    Maternal Health Risk prediction MLOps pipeline

  7. youtube_data_analysis

    Created an optimised pipeline to provide accurate data for analysis, then used snowsight (provided by Snowflake) to create a dashboard.

  8. petastorm

    Petastorm library enables single machine or distributed training and evaluation of deep learning models from datasets in Apache Parquet format. It supports ML frameworks such as Tensorflow, Pytorch, and PySpark and can be used from pure Python code.

  9. AppSignal

    AppSignal knows why the f*#k it crashed. Stop vibe-debugging. Every exception, every backtrace, grouped so you see patterns, not noise.

    AppSignal logo
  10. Prefect

    The easiest way to build, run, and monitor data pipelines at scale.

NOTE: The number of mentions on this list indicates mentions on common posts plus user suggested alternatives. Hence, a higher number means a better weather_data_pipeline alternative or higher similarity.

weather_data_pipeline discussion

Log in or Post with

weather_data_pipeline reviews and mentions

Posts with mentions or reviews of weather_data_pipeline. We have used some of these posts to build our list of alternatives and similar projects.

Stats

Basic weather_data_pipeline repo stats
1
5
4.2
over 3 years ago

Sponsored
Stop Scripting Promotions. Start Shipping with Kargo
Kargo automates promotion across dev, staging, and prod with approval gates and verification. Open source, built by the team behind Argo CD. Download now.
akuity.io

Did you know that Python is
the 1st most popular programming language
based on number of references?