pgbouncer
pg_bulkload
| pgbouncer | pg_bulkload | |
|---|---|---|
| 34 | 4 | |
| 2,648 | 399 | |
| 1.5% | 1.8% | |
| 8.7 | 4.6 | |
| 6 days ago | about 2 months ago | |
| C | C | |
| GNU General Public License v3.0 or later | GNU General Public License v3.0 or later |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
pgbouncer
-
MongoDB and Load Balancer Support
Thanks to MongoDB drivers all consistently providing connection monitoring and pooling functionality, external connection pooling solutions aren't required (ex: Pgpool, PgBouncer). This allows applications built using MongoDB drivers to be resilient and scalable out of the box, but based on what we understand regarding the number of connections applications establish to MongoDB clusters it stands to reason that at a certain point as our application deployments increase, so will our connections.
-
Minha jornada de otimização de uma aplicação django
Pgbouncer - resolvia o problema do limite de conexões no postgres. Mas a API “saudável” manteve o número de conexões baixo o suficiente.
- PgBouncer 1.21.0 – "The one with prepared statements"
- Pgbouncer adds support for prepared statements
-
PgBouncer is useful, important, and fraught with peril
Pgbouncer maintainer here. Overall I think this is a great description of the tradeoffs that PgBouncer brings and how to work around/manage them. I'm actively working on fixing quite a few of the issues in this blog though
1. Named protocol-level prepared statements in transaction mode has a PR that's pretty close to being merged: https://github.com/pgbouncer/pgbouncer/pull/845
-
Supavisor: Scaling Postgres to 1 Million Connections
A common solution is connection pooling. Supabase currently offers pgbouncer which is single-threaded, making it difficult to scale. We've seen some novel ways to scale pgbouncer, but we have a few other goals in mind for our platform.
-
Citus 12: Schema-based sharding for PostgreSQL
Great observation! :)
We worked upstream to have `search_path` properly handled (tracked per client) by pgbouncer.
https://github.com/pgbouncer/pgbouncer/commit/8c18fc4d213ad4...
Check config.md in that commit for a verbose, humanized description.
pg_bulkload
-
Ingesting multiple large .sql files into Postgres database
Did you look at tools like https://github.com/ossc-db/pg_bulkload ?
-
SQLalchemy, postgres, and parallelizing
TLDR I would restrict the use of Python to the data processing and use a dedicated solution for loading the data into Postgres, such as the native COPY on a table without keys/indexes, or pg_bulkload.
- How to import csv to postgresql in a faster way ?
-
MEET A STEERING COMMITTEE MEMBER: TORU SHIMOGAKI OF NTT DATA
Tell us about your career path, interest in green software and your journey to the Green Software Foundation (GSF) As a software engineer, I initially worked on performance improvements and feature additions for PostgreSQL, an open-source software - RDBMS. I am the original developer of a data loading utility called pg_bulkload. After that, I supported the introduction of Apache Hadoop, Apache Spark, and Apache Kafka, which are big data platforms.
What are some alternatives?
odyssey - Scalable PostgreSQL connection pooler
psycopg2 - PostgreSQL database adapter for the Python programming language
asyncpg - A fast PostgreSQL Database Client Library for Python/asyncio.
PipelineDB - High-performance time-series aggregation for PostgreSQL
pgcat - PostgreSQL pooler with sharding, load balancing and failover support. [Moved to: https://github.com/postgresml/pgcat]
citus - Distributed PostgreSQL as an extension
TimescaleDB - An open-source time-series SQL database optimized for fast ingest and complex queries. Packaged as a PostgreSQL extension.
Greenplum - Greenplum Database - Massively Parallel PostgreSQL for Analytics. An open-source massively parallel data platform for analytics, machine learning and AI.
pgcat - PostgreSQL pooler with sharding, load balancing and failover support.
pspg - Unix pager (with very rich functionality) designed for work with tables. Designed for PostgreSQL, but MySQL is supported too. Works well with pgcli too. Can be used as CSV or TSV viewer too. It supports searching, selecting rows, columns, or block and export selected area to clipboard.
rds-auth-proxy - A "passwordless" login experience for your AWS RDS