PostgreSQL
gh-ost
PostgreSQL | gh-ost | |
---|---|---|
57 | 32 | |
11,922 | 12,023 | |
- | 0.7% | |
8.0 | 7.5 | |
7 days ago | about 16 hours ago | |
JavaScript | Go | |
MIT License | MIT License |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
PostgreSQL
-
Neon Is Generally Available: Serverless Postgres
pg doesn't do too well with serverless, dead connections are left in the pool (or something)
https://github.com/brianc/node-postgres/issues/2112
-
NodeJS Security Best Practices
If you don't want to use ORM then there are some other packages as well! For PostgreSQL we have node-postgres
-
Building Secure Neon-Infused Web Apps with Auth0, Express, and EJS
Interface with PostgreSQL database
-
Drizzle is just as unready for prime-time as Prisma, what else is there?
(Instead of the following with pg.)
-
Nile, Serverless Postgres for Modern SaaS
So far every JS framework that uses https://node-postgres.com works great and so no reason to think Drizzle wouldn't.
-
We migrated to SQL. Our biggest learning? Don't use Prisma
One thing that keeps coming up is that SQL equals low productivity. I don't think this is true. I think the culprit is that most developers are using to heavily abstracting SQL using ORMs like Prisma that hides the database and SQL logic.
Since building a SQL generator (https://aihelperbot.com) as a side project, I have become much more proficient in SQL and even though I am also locked into Prisma, I use the `queryRaw` all the time to execute raw SQL queries. You can understand the code without knowing Prisma API. It is more performant. For more complex SQL queries, I use the SQL generator for initial suggestions and adapt if needed.
For the next projects I build I want to use the minimal Postgres client (https://github.com/brianc/node-postgres) combined with a lightweight migration library.
-
Using AI I have departed from ORM and embraced SQL
For newer projects I use the small Postgres client. Initially my leap into SQL was lead by AI but as I refreshed and relearned SQL, I now use a mixture of AI and self-written SQL queries. Something like this is just easier to have AI do the grunt work and then adjustment as needed.
-
Credentials Leak with Knex
This was a known issue for pg developers, and they managed to fix it a long time ago (at the pg level), but the knowledge of this problem didn't reach Knex maintainers.
-
Why SQL is right for Infrastructure Management
Integrate the database into your application itself with a postgres client library allowing your applications to make infrastructure changes (like provisioning sharded resources for a client that wants isolation, or using a more accurate forecasting model to pre-allocate more resources before the storm hits).
-
What is your development stack for 2023?
node-postgres (raw sql, without ORM)
gh-ost
- "At GitHub we do not use foreign keys, ever, anywhere"
-
How Modern SQL Databases Are Changing Web Development - #3 Better Developer Experience
I’ve been through multiple incidents where everything worked fine in the testing environment but ended up locking the production database for minutes when deployed. A category of open-source tools called OSC (Online Schema Change) exists to mitigate such pain, like gh-ost used by GitHub and OSC used by Meta. They work by creating a set of "ghost tables" to apply the migrations, copy over old data from the original tables, and catch up with new writes simultaneously. When all old data is migrated, you can trigger a cutover to make the "ghost tables" production. Check the post below for a great introduction and comparison:
-
We migrated to SQL. Our biggest learning? Don't use Prisma
Sounds like it's basically explained in the gh-ost readme https://github.com/github/gh-ost#how
I think it amounts to "use views to decouple access to the table with a fixed interface" and "use triggers for migrating data between tables"
-
Ask HN: Is PostgreSQL better than MySQL?
Gh-ost is the new hotness. Simple to use and lots of great features: https://github.com/github/gh-ost
-
My Green/blue AWS db deployment strategy for avoiding data loss due to table locks
If the performance of the db is a concern during migrations (locking, high cpu consumption for large writes) there are tools that can help and do similiar to what your describing but with the benefit that they are battle tested tools. This one spring to mind https://github.com/github/gh-ost there are other options as well and its worth reading the trade off docs
-
Changing column from longtext to mediumtext taking over 2 hours
Not sure which version of MySQL you're using, but one approach would be to use a tool like pt-online-schema-change (from Percona) or g-host -- which will create a duplicate table and then swap it in place of the original table. It's a safer approach when operating in production environments. Here's a good comparison of the tools many people use https://planetscale.com/docs/learn/online-schema-change-tools-comparison
-
Ask HN: Do you use foreign Keys in Relational Databases
No, especially on large tables with billions of records. They make online schema changes impossible. More details: https://github.com/github/gh-ost/issues/331#issuecomment-266...
-
Migrating a production database without any downtime
Tip #4: Consider slow-running migrations. Some tables can be so large that the traditional migration way is simply not a viable option for them. In such cases, you can consider embedding the data migration code right into your application, or use a special utility like GitHub's online schema migration for MySQL. A slow-running migration can work in production for days or even weeks. It gradually converts the data by small chunks, so you can carefully balance the load on the database while making sure that it doesn't cause slowness or downtime.
-
How do you handle RDS schema migrations?
GitHub gh-ost
-
Changing Tires at 100mph: A Guide to Zero Downtime Migrations
Actually I never tried but I was scared by the small print of GH not using RDS themselves [1] and Ghost relying on lower-level features that might be not easily available in RDS. Also I had the impression you have to setup a normal non-RDS replica attached to your RDS master?
[1] https://github.com/github/gh-ost/blob/master/doc/rds.md
What are some alternatives?
Prisma - Next-generation ORM for Node.js & TypeScript | PostgreSQL, MySQL, MariaDB, SQL Server, SQLite, MongoDB and CockroachDB
pg-online-schema-change - Easy CLI tool for making zero downtime schema changes and backfills in PostgreSQL [Moved to: https://github.com/shayonj/pg-osc]
MySQL - A pure node.js JavaScript Client implementing the MySQL protocol.
doctrine-test-bundle - Symfony bundle to isolate your app's doctrine database tests and improve the test performance
TypeORM - ORM for TypeScript and JavaScript. Supports MySQL, PostgreSQL, MariaDB, SQLite, MS SQL Server, Oracle, SAP Hana, WebSQL databases. Works in NodeJS, Browser, Ionic, Cordova and Electron platforms.
squawk - 🐘 linter for PostgreSQL, focused on migrations
MongoDB - The official MongoDB Node.js driver
pg_squeeze - A PostgreSQL extension for automatic bloat cleanup
Aerospike - Node.js client for the Aerospike database
hub - A command-line tool that makes git easier to use with GitHub.
Redis - 🚀 A robust, performance-focused, and full-featured Redis client for Node.js.
Jenkins - Jenkins automation server