Invitation to collaborate on open source PySpark projects

This page summarizes the projects mentioned and recommended in the original post on /r/apachespark

Judoscale - Save 47% on cloud hosting with autoscaling that just works
Judoscale integrates with Django, FastAPI, Celery, and RQ to make autoscaling easy and reliable. Save big, and say goodbye to request timeouts and backed-up task queues.
judoscale.com
featured
InfluxDB high-performance time series database
Collect, organize, and act on massive volumes of high-resolution data to power real-time intelligent systems.
influxdata.com
featured
  1. quinn

    pyspark methods to enhance developer productivity 📣 👯 🎉 (by mrpowers-io)

    quinn is a library with PySpark helper functions. I need to work through all the open issues / PRs and bump all versions. I should do another release. This library gets around 600,000 monthly downloads.

  2. Judoscale

    Save 47% on cloud hosting with autoscaling that just works. Judoscale integrates with Django, FastAPI, Celery, and RQ to make autoscaling easy and reliable. Save big, and say goodbye to request timeouts and backed-up task queues.

    Judoscale logo
  3. chispa

    PySpark test helper methods with beautiful error messages

    chispa is a library of PySpark testing functions.

  4. dat

    Delta Acceptance Testing (by delta-incubator)

    Delta Acceptance Testing (dat) is a library that creates Delta Lake reference tables. This project is being done with the core Delta Lake devs. We need to build out all the reference tables and write tests to make sure PySpark can fully implement the Delta Lake protocol.

NOTE: The number of mentions on this list indicates mentions on common posts plus user suggested alternatives. Hence, a higher number means a more popular project.

Suggest a related project

Related posts

  • Pyspark now provides a native Pandas API

    3 projects | /r/Python | 2 Jan 2022
  • Why Databricks Is Winning

    5 projects | news.ycombinator.com | 14 Feb 2021
  • Testing spark applications

    1 project | /r/dataengineering | 5 Jul 2023
  • Brainstorming functions to make PySpark easier

    1 project | /r/apachespark | 13 Mar 2023
  • PySpark OSS Contribution Opportunity

    3 projects | /r/apachespark | 5 Mar 2023

Did you know that Python is
the 2nd most popular programming language
based on number of references?