sqitch VS migrate

We use https://sqitch.org/ and we’re fairly happy with it. Sqitch manages the files to deploy which are applied fits to a local database.

We use GitHub actions for deployment and database migrations are just one step of the pipeline. The step invokes sqitch deploy which runs all the pending migration files.

Then, all the approval process is standard for the environment. We require approvals in pull requests before merging to the main branch.

I'm experimenting with it right now using Squitch [1] to make maintenance easier. It still feels like a hack and I also still have my doubts about the viability of this for real-world use. It's fun though and I'm learning about all kinds of advanced Postgres features.

[1] https://sqitch.org/

For developing I find the official Perl docker images, running on a lightweight version of Debian, to be perfectly fine. Later on you might hand roll the skinniest possible image but the beauty of this setup is you can do that later and you don't need to change anything else. There's really not a lot going on here. First I declare the base image, which is as I said the official Perl image. I'm not using the latest Perl here because the application uses Sqitch for managing database migrations and that needs an update (there's a PR pending) to run on the most recent Perl so we'll just use a very nearly recent one instead. WORKDIR just defines where your application is installed. You can put it anywhere you want within reason. I like simple things so I use the most simple of all the conventions I've seen around.

Also, https://sqitch.org/

Some possible upsides of codd: - No need to manually write verification SQL. Codd will update schema representation files when you codd add some-migration.sql and will compare those to the actual schema when deploying (I'd say in ways which would be very hard to replicate manually, see an example of what codd checks, giving you the option to rollback if they don't match or proceed but log non-matching db objects. - It seems to be much simpler to set codd up. You need 3 env vars to start, a folder to store your migrations and a self-contained statically linked executable. Just codd add migration.sql your way in after that - This might be very wrong as I couldn't find it explicitly documented, but this GH issue suggests it's not so simple to apply all pending migrations in a single transaction with Sqitch? Maybe it requires some bundling or something along those lines and then it's fine, though. In any case, codd will do this automatically when you run codd up (provided postgresql allows it).

Go does not natively support the use of migrations, but we could use the ORM that has this functionality, such as GORM which is the most used by the community, but We can use migrations without using an ORM, for this we will use the golang-migrate package.

$ curl -L https://github.com/golang-migrate/migrate/releases/download/$version/migrate.$os-$arch.tar.gz | tar xvz

Golang-migrate is a database migration tool designed for Go applications. It helps manage and apply changes to the database schema as the application grows, ensuring that the code and database structure stay in sync.

First of all, thank you for SQLAlchemy! If I ever had to make a final choice in how I would interact with a database for a very large project that involves a considerable dev team, I would always bet on SQLAlchemy. Not that I would necessarily like all aspects of it, but when it comes to Python and SQL - “Nobody ever got fired for picking SQLAlchemy.”.

With that out of the way, despite ORMs doing much more than "just writing SQL", it is exactly on that point that I flinch: Most devs should be exposed to SQL. And if your project allows you to build around simple enough abstractions so that you aren't reinventing the wheel, you should definitely be writing SQL. Especially if you don't know SQL yet - which is the growing case of new devs coming into the job market.

You can achieve a lot with SQlAlchemy Core, a tool that I absolutely recommend, but my post is just a simple alternative to get developers to think about their approach. If that results in some devs reconsidering using "full fat" SQLAlchemy and to try SQLAlchemy Core, that's a win for me!

Your gist tries to highlight the difficulty of doing certain things without an ORM. Migrations (as just 1 example) doesn't need to be hard, simple tools like flyway, or migrate (https://github.com/golang-migrate/migrate) achieve a similar result (while also keeping you on the path of writing SQL!). Deep and complex relationships between objects also don't need to be hard - typically people approach this subject with a requirement to be very flexible in the way they want to build queries and objects, but that to me in a sign that maybe they should reconsider their business logic AND reconsider that, just maybe, their project doesn't require all that flexibility, it is fairly straightforward to extend objects and introduce some more complex representations as and when it is needed - will all of this make me write code faster? Absolutely not. That is why you have spent so much time perfecting SQLAlchemy, but then again, I am not advocating for devs to go and replace their usage of ORMs, just presenting an alternative that may or may not fit their needs for a new project + give devs the chance to learn something that the ORM might have taken away.

Doing this now with a mysql db for my use case. Using sp to take a large chunk of data migration load off my data layer code. I am using migrate (go library) for migrations and hooked it up with a bunch of test suites for all SP and Triggers it creates. I test it against a testDB maintained as part of my CI/CD. Haven’t had an issue with production yet. It does however require quite a bit of initial setup.

Before we can start using MySQL we need to create a table to store our data. I will be using excellent migrate database migrations tool, it can also be imported as a libraray.

We need a database table to store our application's users' data. To generate and migrate a schema, we'll use golang migrate. Kindly follow these instructions to install it on your Operating system. To create a pair of migration files (up and down) for our user table, issue the following command in your terminal and at the root of your project: