Outgrowing Postgres: Handling increased user concurrency

1. scylla-javascript-driver

   1 2 52 8.9 Rust

   A ScyllaDB Driver alternative for Nodejs, Javascript and Typescript written using Node API through Rust bindings

   

   NoSQL: For certain types of data and access patterns, a NoSQL database like MongoDB, ScyllaDB, or DynamoDB might be more suitable for high-concurrency scenarios, as long as your data makes more sense being denormalized.

2. CodeRabbit

   coderabbit.ai featured

   CodeRabbit: AI Code Reviews for Developers. Revolutionize your code reviews with AI. CodeRabbit offers PR summaries, code walkthroughs, 1-click suggestions, and AST-based analysis. Boost productivity and code quality across all major languages with each PR.

   

3. RabbitMQ

   2 114 12,633 10.0 JavaScript

   Open source RabbitMQ: core server and tier 1 (built-in) plugins

   

   Asynchronous processing can be implemented using queueing technologies like RabbitMQ or Apache Kafka. For Python applications, a popular go-to is Celery, a distributed task queue that integrates well with various web frameworks.

4. pgbouncer

   3 36 3,197 8.7 C

   lightweight connection pooler for PostgreSQL

   

   Connection pooling is usually the first and most effective fix to increase connection capacity. PgBouncer addresses two key scalability constraints in Postgres: per-connection memory overhead and context switching costs. Instead of Postgres maintaining separate processes and memory allocations, PgBouncer manages a small pool of actual Postgres connections that serve a larger number of client connections by assigning requests to available connections from the pool. This eliminates the overhead of repeatedly setting up and closing connections.

5. MongoDB

   4 298 26,929 10.0 C++

   The MongoDB Database

   

   NoSQL: For certain types of data and access patterns, a NoSQL database like MongoDB, ScyllaDB, or DynamoDB might be more suitable for high-concurrency scenarios, as long as your data makes more sense being denormalized.

6. cockroach

   5 111 30,632 10.0 Go

   CockroachDB — the cloud native, distributed SQL database designed for high availability, effortless scale, and control over data placement.

   

   Postgres-compatible DSQL solutions: Distributed SQL databases that are either Postgres or Postgres-compatible like Citus, Aurora DSQL, and CockroachDB might be the right approach for handling your scale without forcing you to change much of your code. They each have their strengths and weaknesses and the right one for you depends on your use cases and needs.

7. citus

   6 71 11,167 8.2 C

   Distributed PostgreSQL as an extension

   

   Postgres-compatible DSQL solutions: Distributed SQL databases that are either Postgres or Postgres-compatible like Citus, Aurora DSQL, and CockroachDB might be the right approach for handling your scale without forcing you to change much of your code. They each have their strengths and weaknesses and the right one for you depends on your use cases and needs.

8. Sidekiq

   7 97 13,272 9.5 Ruby

   Simple, efficient background processing for Ruby

   

   If you’re developing in Node, BullMQ has been rising in popularity as a go-to solution. For Rails applications, you can use ActiveJob with backends like Sidekiq for efficient background job processing.

9. SaaSHub

   www.saashub.com featured

   SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives

   

10. Redis

    8 368 68,399 9.7 C

    Redis is an in-memory database that persists on disk. The data model is key-value, but many different kind of values are supported: Strings, Lists, Sets, Sorted Sets, Hashes, Streams, HyperLogLogs, Bitmaps.

    

    A caching layer using popular in-memory databases like Redis or Memcached can go a long way in addressing Postgres connection overload issues by being able to handle a much larger concurrent request load. Adding a cache lets you serve frequent reads from memory instead, taking pressure off Postgres.

11. pinot

    9 18 5,683 9.9 Java

    Apache Pinot - A realtime distributed OLAP datastore

    

    Specialized databases: For analytical workloads, consider using a dedicated analytical database optimized for high-concurrency, low-latency queries on large datasets like Pinot, DuckDB, or ahem Tinybird.

12. PostgreSQL

    10 58 12,552 8.4 JavaScript

    PostgreSQL client for node.js.

    

    Here's a sample Typescript implementation using node-postgres, a popular collection of tools for working with Postgres in Node:

13. Next.js

    11 2,293 130,389 10.0 JavaScript

    The React Framework

    

    If you’re developing a web application, many popular frameworks make it easy to implement request and router caching, reducing the number of requests that hit your backend and subsequently, your Postgres instance(s). For NextJS developers, the Data Cache, Full Route Cache, and Router Cache are great options for caching across several layers of your application, abstracting away many of the underlying implementation details, allowing you to focus on your application logic.

14. Memcached

    12 63 13,754 8.9 C

    memcached development tree

    

    A caching layer using popular in-memory databases like Redis or Memcached can go a long way in addressing Postgres connection overload issues by being able to handle a much larger concurrent request load. Adding a cache lets you serve frequent reads from memory instead, taking pressure off Postgres.

15. ApacheKafka

    13 129 35 0.0

    A curated re-sources list for awesome Apache Kafka

    

    Asynchronous processing can be implemented using queueing technologies like RabbitMQ or Apache Kafka. For Python applications, a popular go-to is Celery, a distributed task queue that integrates well with various web frameworks.

Suggest a related project