Top 9 Python pydata Projects

pydata

• Add a project

1. Dask

   1 33 12,857 9.6 Python

   Parallel computing with task scheduling

   

   Project mention: Ask HN: What's the right tool for this job? | news.ycombinator.com | 2024-07-20

   From what I've seen, there are sort of two paths. I'll provide a well known example from each.

   1. lang specific distributed task library

   For example, in Python, celery is a pretty popular task system. If you (the dev) are the one doing all the code and running the workflows, it might work well for you. You build the core code and functions, and it handles the processing and resource stuff with a little config.

   * https://github.com/celery/celery

   Or lower level:

   * https://github.com/dask/dask

   2. DAG Workflow systems

   There are also whole systems for what you're describing. They've gotten especially popular in the ML ops and data engineering world. A common one is AirFlow:

   * https://github.com/apache/airflow

2. SaaSHub

   www.saashub.com featured

   SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives

   

3. stumpy

   2 8 3,734 8.4 Python

   STUMPY is a powerful and scalable Python library for modern time series analysis

   

   Project mention: Powerful and scalable Python library for modern time series analysis | news.ycombinator.com | 2024-08-01

4. koalas

   3 2 3,346 4.6 Python

   Koalas: pandas API on Apache Spark

   

5. pandas-datareader

   4 3 2,993 4.5 Python

   Extract data from a wide range of Internet sources into a pandas DataFrame.

   

6. distributed

   5 3 1,589 9.5 Python

   A distributed task scheduler for Dask

   

7. pyjanitor

   6 4 1,385 8.6 Python

   Clean APIs for data cleaning. Python implementation of R package Janitor

   

8. Bodo

   7 2 176 9.9 Python

   High-Performance Python Compute Engine for Data and AI

   

   Project mention: Github's Top 31 items of Dec 18, 2024 | dev.to | 2024-12-18

   Github URL: https://github.com/bodo-ai/Bodo

9. pyvtreat

   8 1 122 4.9 Python

   vtreat is a data frame processor/conditioner that prepares real-world data for predictive modeling in a statistically sound manner. Distributed under a BSD-3-Clause license.

   

10. graphblas-algorithms

    9 3 79 0.0 Python

    Graph algorithms written in GraphBLAS

    

Python pydata related posts

• Shuffling large data at constant memory in Dask

  1 project | /r/Python | 17 Apr 2023

• My new company uses Pyspark. I want to learn it before my starting date. Any advice?

  1 project | /r/datascience | 10 Nov 2022

• Great forward progress on squashing cluster deadlocks

  1 project | /r/dask | 15 Dec 2021

• Is Numpy always more efficient than Pandas? And how much should we rely on Python anyway?

  1 project | /r/datascience | 10 Dec 2021

• Ask HN: Is PySPark a Dead-End?

  1 project | news.ycombinator.com | 5 Dec 2021

• How to load 85.6 GB of XML data into a dataframe

  1 project | /r/pythontips | 1 Dec 2021

• How to load 85.6 GB of XML data into a dataframe

  1 project | /r/learnprogramming | 1 Dec 2021
• A note from our sponsor - SaaSHub
  www.saashub.com | 19 Jan 2025

  SaaSHub helps you find the best software and product alternatives Learn more →

Index

Sponsored

SaaSHub - Software Alternatives and Reviews

SaaSHub helps you find the best software and product alternatives

www.saashub.com

Do not miss the trending Python projects with our weekly report!