spawn-demo VS testcontainers-go

Full disclosure way up front: I'm one of the developers working on https://spawn.cc at Redgate and the two scenarios you describe are some examples of exactly what we've built Spawn for.

Not trying to sell you anything (since we are still in beta anyway!) but I'd encourage you to try it out and see if it does what you're looking for. We're in the early stages so any and all feedback is really helpful!

It's a hosted service where you can immediately spin up copies of "data images" (a snapshot of schema and data in a database instance) instantly regardless of the size. As an example, we created a copy of the 400GB public stack overflow DB for development in ~15s.

You can also snapshot the state of the DB at any time and move around those save points whenever you like. Creating a "data image" for later copies is also possible based off the current state of the database, so you can curate data through your web UI, save the state, and then make that dataset available for multiple copies across your dev team.

We've got a couple of repos to show it in action:

- https://github.com/red-gate/spawn-demo

- https://github.com/red-gate/flyway-spawn-demo/actions

As well as a youtube video: https://www.youtube.com/watch?v=sxHtca85CoE

Let's first look at the original version of the test, in this case, an end-to-end, using testcontainers.

It supports multiple languages (including Go!) and databases (also messaging infrastructure etc.) - All you need is Docker. Testcontainers for Go makes it simple to programmatically create and clean up container-based dependencies for automated integration/smoke tests. You can define test dependencies as code, run tests and delete the containers once done.

Testcontainers Go Documentation

Some of the features: - ✅ Using Vertical Slice Architecture as a high level architecture - ✅ Using Event Driven Architecture on top of RabbitMQ Message Broker with a custom [Event Bus](pkg/messaging/bus/) - ✅ Using Event Sourcing in Audit Based services like [Orders Service](services/orders/) - ✅ Using CQRS Pattern and Mediator Patternon top of Go-MediatR library - ✅ Using Dependency Injection and Inversion of Controlon top of uber-go/fx library - ✅ Using RESTFul api with Echo framework and using swagger with swaggo/swag library - ✅ Using Postgres and EventStoreDB to write databases with fully supports transactions(ACID) - ✅ Using MongoDB and Elastic Search for read databases (NOSQL) - ✅ Using OpenTelemetry for collection Distributed Tracing with using Jaeger and Zipkin - ✅ Using OpenTelemetry for collection Metrics with using Prometheus and Grafana - ✅ Using Unit Test for testing small units with mocking dependent classes and using Mockery for mocking dependencies - ✅ Using End2End Test and Integration Test for testing features with all of their real dependeinces using docker containers (cleanup tests) and testcontainers-go library

For example, you could take a look at this open-source project that helps you spawn docker containers as part of your test setup enabling you to execute queries to a local database.

Using something like TestContainers to spin up a DB for testing has been my best experience. Any mocks leave too much room for error imo

I highly recommend testcontainers for this.

The next block in .gitlab-ci.yml is the services block. Since our tests use testcontainers package and we're pushing a docker container onto Dockerhub, we will need to specify a services block next. Services will enable our pipeline to leverage Docker-in-Docker DinD.