Top 10 Partitioning Open-Source Projects

pg_partman

7 1,873 7.3 PLpgSQL

Partition management extension for PostgreSQL

Project mention: Dear data engineers | /r/dataengineering | 2023-12-06

Assuming these are the types of insights you're looking for, you'll probably look for a way to aggregate data points across/within geographies. postgis is an open source extension for postgres that can help you with this, but there's also quite a few python tools that can help you explore the data, such as geopandas, folium, geoplot. Depending on volume, you might want to partition the data for query performance, and there's another extension pg_partman that can help with that. Just noticed some other posts have recommended something similar.

pg_party

1 442 3.8 Ruby

ActiveRecord PostgreSQL Partitioning
InfluxDB

www.influxdata.com sponsored

Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
CXXGraph

84 393 8.5 C++

Header-Only C++ Library for Graph Representation and Algorithms

Project mention: Hacktoberfest is ON CXXGraph | dev.to | 2023-10-09

Actual Web Site

shards

0 292 1.8 Erlang

Partitioned ETS tables for Erlang and Elixir (by cabol)
zabbix-mysql-partitioning-perl

3 30 4.1 Perl

This script is a script written in Perl to partition the Zabbix database tables in time based chunks. We can use this script to replace the Zabbix housekeeper process which tends to get too slow once you hit a certain database size.
segmenter

1 17 4.3 Go
scikit-multilearn-ng

1 16 7.9 Python

A new maintained "successor" to scikit-multilearn, a scikit-learn based module for multi-label et. al. classification

Project mention: I made a follow-up package, scikit-multilearn-ng, to the widely used scikit-multilearn package for multilabel classification | dev.to | 2024-01-29

So after some developing, I'm here to introduce scikit-multilearn-ng (GitHub: https://github.com/scikit-multilearn-ng/scikit-multilearn-ng), an advanced, open-source tool for multi-label classification in Python. It's a direct successor to scikit-multilearn and brings a host of improvements and new features.

WorkOS

workos.com sponsored

The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.
kafka

1 14 10.0 TypeScript

Realtime BTC and ETH Wallet Tracker with Node.js and Kafka (by flolu)
PgDice

0 10 0.0 Ruby

Postgres partitioning built on top of https://github.com/ankane/pgslice
Install11

2 2 10.0 Batchfile

A script that allows you to install Windows 11 by skipping OOBE and other pain

NOTE: The open source projects on this list are ordered by number of github stars. The number of mentions indicates repo mentiontions in the last 12 Months or since we started tracking (Dec 2020).