wal-e VS postgres-operator

For me, it's case-by-case. I don't back up the VMs directly, just the date of the stateful applications running on the VMs (or bare metal servers, I do identical stuff for them).

For postgres, I used to just have a systemd timer that would `pg_dumpall` and throw it in s3.

Now I use https://github.com/wal-e/wal-e to backup my postgresql databases.

For other local files, I use borg backup for personal files and services I just run for myself, and I use restic to backup server files to s3.

The operating system's configuration is all stored in git via the magic of NixOS, so I don't have to worry about files in /etc, they all are 100% reproducible from my NixOS configuration.

See the GitHub: https://github.com/wal-e/wal-e

Unmaintained would’ve made more sense to say, but the maintainer choose the words “obsolete” so I took those. :)

Seems to be obsolete due to a lack of interest and contributions.

I believe the idea originally started with Wal-E project (https://github.com/wal-e/wal-e) since then there were many improvements to it and AWS no doubt has their own proprietary solution.

Hope this helps someone out there.

https://github.com/wal-e/wal-e

Running your own postgres server on AWS is not difficult. You'll want to get familiar with EBS and look into using a tool like wal-e to manage your WAL logs and ensure yuor ability to recover from the various exciting failure modes that Amazon makes available to you.

These read-replicas leveraged WAL segments created by the primary database; however, we didn’t want to bog down the primary database with each replica by reading directly from production. To circumvent this issue, we leverage wal-e, a tool that performs continuous archiving of PostgreSQL WAL files and base backups, and read replicas restored from s3 or gcs versus reading directly from the primary database.

Have a look at barman, pgBackrest or for maybe WAL-G or WAL-E

I am using the following docker image https://github.com/docker-library/postgres. I want the ability to make regular backups to my S3 bucket. PITR is not necessary but a nice to have. I've tried to use https://github.com/wal-e/wal-e but I've not found something that works very well with docker.

There are two major ways to deploy Postgres to Kubernetes. First is using Helm charts like bitnami/postgresql or Serge’s postgresql-single, which has benefits compared to bitnami chart. The second way to run Postgres is to use the Kubernetes operator. There are a couple of them, here we will talk about Zalando Postgres operator. The helm way is better when in the future you will not plan to use more than one or two Postgres clusters per Kubernetes cluster. If you realize, that you will run more, then the Operator way is for you with all its automation and management advantages. Here is an example of PostgreSQL database manifest for Zalando Postgres Operator pg-prod.yaml. This example was tested for intensive transaction processing with query rates up to 3000-4000 per second and database size up to 350GB (data and indexes). It is enough to start most projects with thousands of active users like medium-volume webshops, news- and community-support websites, medium-sized gaming or gambling, etc. We assume, that we already set up the operator and the S3 bucket with encryption and access control for backups and WAL-log storage.

yes, precisely. It's UI part that's broken, which cannot list snapshots. Issue is here, no fix since 2020, sadly: https://github.com/zalando/postgres-operator/issues/937

I personally like the Zalando operator better, you can add databases and users by updating the CRD, feature parity between the two on HA is pretty good

https://github.com/zalando/postgres-operator

one of the solutions which made it pretty simple for us to run postgresql in a ha environment (mostly in k8s, but works standalone as well) is zalandos patroni: https://github.com/zalando/patroni it's really solid and worked for us for a few years already.

or for k8s their operator: https://github.com/zalando/postgres-operator (docker image: https://github.com/zalando/spilo) we've also tried other operators which were easier to get started, but they failed miserably (crunchyrolls operator is basically based on the zalando one)

git clone https://github.com/zalando/postgres-operator.git cd postgres-operator

Zalando / Postgres-Operator

Somewhere between here and here i found out about that.

Here's an example of using the PostgreSQL Operator to deploy a high availability PostgreSQL cluster in Kubernetes. In this example, I'll be using the PostgreSQL Operator from Zalando [https://github.com/zalando/postgres-operator] to deploy a PostgreSQL cluster with two nodes.