Weekend Data Engineering Project-Building Spotify pipeline using Python and Airflow. Est.Time:[4–7 Hours]

This page summarizes the projects mentioned and recommended in the original post on /r/dataengineering

Our great sponsors
  • WorkOS - The modern identity platform for B2B SaaS
  • InfluxDB - Power Real-Time Data Analytics at Scale
  • SaaSHub - Software Alternatives and Reviews
  • Spotify_etl

  • You may be aware, but they’ll stay visible in your commit history. You’ll want to invalidate the token if you haven’t already.

  • spotify-api

    Pipeline that extracts data from the Spotify API to build a more detailed version of Spotify Wrapped

  • Was not expecting to see my project listed here lol. I feel honored. If anyone is interested here's the project link where I recreated a more detailed version of Spotify Wrapped: https://github.com/calbergs/spotify-api. Always looking to improve upon this as well so happy to receive any feedback for anyone that comes across this.

  • WorkOS

    The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.

    WorkOS logo
  • sops

    Simple and flexible tool for managing secrets

  • I'd suggest using something like SOPS for a project like this. Using more specific solutions is preferable over bringing in a technology like Ansible just to use its vault.

NOTE: The number of mentions on this list indicates mentions on common posts plus user suggested alternatives. Hence, a higher number means a more popular project.

Suggest a related project

Related posts