Our great sponsors
-
dockertest
Write better integration tests! Dockertest helps you boot up ephermal docker images for your Go tests with minimal work.
-
testcontainers-node
Testcontainers is a NodeJS library that supports tests, providing lightweight, throwaway instances of common databases, Selenium web browsers, or anything else that can run in a Docker container.
-
InfluxDB
Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
-
kubedock
Kubedock is a minimal implementation of the docker api that will orchestrate containers on a Kubernetes cluster, rather than running containers locally.
-
phoenix_container_example
Production-quality example for Elixir/Phoenix building, testing, and running in containers
-
Testcontainers
Testcontainers is a Java library that supports JUnit tests, providing lightweight, throwaway instances of common databases, Selenium web browsers, or anything else that can run in a Docker container.
-
WorkOS
The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.
-
embedded-postgres-binaries
Lightweight bundles of PostgreSQL binaries with reduced size intended for testing purposes.
-
pglite
Lightweight Postgres packaged as WASM into a TypeScript library for the browser, Node.js, Bun and Deno
-
nammayatri
A Direct-to-Driver open mobility platform powering the next-generation of mobility applications in India.
-
gradle-docker-compose-plugin
Simplifies usage of Docker Compose for integration testing in Gradle environment.
-
SaaSHub
SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives
I've been using Docker containers for integration testing for last few years. I usually roll my own custom solution in Go using the https://github.com/ory/dockertest package though that adds necessary functionality around running migrations, creating kafka topics, or similar. Will definitely need to check out this next time I'm writing a test package
Test containers is such a game changer for integration testing, they have language specific docker apis that make it trivial to bring up containers and verify that they are ready doe testing.
Pretty much every project I create now has testcontainers for integration testing :)
I setup CI so it lints, builds, unit tests then integration tests (using testcontainers)
https://github.com/turbolytics/latte/blob/main/.github/workf...
Their language bindings provide nice helper functions for common database operations (like generating a connection uri from a container user)
https://github.com/turbolytics/latte/blob/main/internal/sour...
We use [kubedock](https://github.com/joyrex2001/kubedock) to run testcontainers in kubernetes clusters. As long as you're only pulling the images, not building or loading them (explicitly not supported by kubedock), it works pretty well.
It's particularly useful for testing a set of microservices.
See https://github.com/cogini/phoenix_container_example for a full example.
except the parent is wrong (at least the Java impl). see:
https://github.com/testcontainers/testcontainers-java/blob/m...
https://github.com/testcontainers/testcontainers-java/blob/m...
> GHA has "service containers", but unfortunately the feature is too basic to address real-world use cases: it assumes a container image can just … boot! … and only talk to the code via the network. Real world use cases often require serialized steps between the test & the dependencies, e.g., to create or init database dirs, set up certs, etc.)
My biased recommendation is to write a custom Dagger function, and run it in your GHA workflow. https://dagger.io
If you find me on the Dagger discord, I will gladly write a code snippet summarizing what I have in mind, based on what you explained of your CI stack. We use GHA ourselves and use this pattern to great effect.
Disclaimer: I work there :)
I'm surprised this is getting so much attention. I thought this just standard practice at this point? If you use things like Gitlab CI then you get this via the `services` in your pipeline. The CI job itself runs in a container too.
I use a very similar thing via pytest-docker: https://github.com/avast/pytest-docker The only difference seems to be you declare your containers via a docker-compose file which I prefer because it's a standard thing you can use elsewhere.
If your project uses Nix, checkout services-flake for running services via Nix.
https://github.com/juspay/services-flake
We actually do this in Nammayatri, an OSS project providing "Uber" for autos in India.
https://github.com/nammayatri/nammayatri
There is a services-flake module allowing you to spin the entire nammayatri stack (including postgres, redis, etc.) using a flake app. Similarly, there's one for running load test, which is also run in Jenkins CI.
If your project uses Nix, checkout services-flake for running services via Nix.
https://github.com/juspay/services-flake
We actually do this in Nammayatri, an OSS project providing "Uber" for autos in India.
https://github.com/nammayatri/nammayatri
There is a services-flake module allowing you to spin the entire nammayatri stack (including postgres, redis, etc.) using a flake app. Similarly, there's one for running load test, which is also run in Jenkins CI.
Something that improved developer experience by far and also sped up our builds is starting the container dependencies via docker-compose and connect to it for integration testing. This allows reuse of containers, you can connect to it after/during an integration test to debug without having to keep searching for ports constantly.
With TestContainers - I've perceived that running integration tests / a single test repeatedly locally is extremely slow as the containers are shut down when the java process is killed. This approach allows for this while also allowing to keep it consistent - example, just mount the migrations folder in the start volume of your DB container and you have a like-for-like schema of your prod DB ready for integration tests.
I've found the https://github.com/avast/gradle-docker-compose-plugin/ very useful for this.
Anyone have an opinion of embedded-postgres vs https://github.com/opentable/otj-pg-embedded (of which its a fork) for Clojure use?
Related posts
- Run and test DynamoDB applications locally using Docker and Testcontainers
- Integration Tests with In Memory DB vs Real DB on Docker
- Unit Testing
- Running untrusted (user-provided) Python code on ASP.NET/C# backend
- If i want to do testing CRUD should I use in memory or just do integration test where I use a seperate database?