dbt-unit-testing VS testcontainers-dotnet

If you use dbt for transformations Dbt Unit Testing (https://github.com/EqualExperts/dbt-unit-testing) is getting some attention (https://www.thoughtworks.com/radar/languages-and-frameworks?blipid=202304042)

dbt-unit-testing

We use this and take an example-based tests approach for any non-trivial tables: https://github.com/EqualExperts/dbt-unit-testing

> How do you test some SQL logic in isolation?

I do this using sql

1. Extracting an 'ephemeral model' to different model file

2. Mock out this model in upstream model in unit tests https://github.com/EqualExperts/dbt-unit-testing

3. Write unit tests for this model.

This is not different than regular software development in a language like java.

I would argue its even better better because unit tests are always in tabular format and pretty easy to understand. Java unit tests on other hand are never read by devs in practice.

I haven't done it yet but there are some popular blogs as well as a DBT package someone created.

> I really don’t understand the communities obsession with unwieldy tools like DBT.

It lets me write test first sql transforms. I never thought TDD sql would be possible. My sql is so much more readable with common logic extracted into ephmeral models. I practice same method to write clear code to write sql, eg: too many mocks = refactor into separate model ( class) .

I think DBT made this possible with refs that can be swapped out with mocks. This is the awesome library I am using https://github.com/EqualExperts/dbt-unit-testing

Testcontainers

Like others said, it's better to test with an actual database. I recommend using Testcontainers (https://dotnet.testcontainers.org), you can even create multiple instances so your tests can run in parallel independently.

To launch a container in code we will use Testcontainers. Testcontainers is a library that is built on top of the .NET Docker remote API and provides a lightweight implementation to support your test environment in all circumstances. This library supports pre-defined packages for containers or you can use your .dockerfile. We will use a pre-defined package for LocalStak. LocalStack is a cloud service emulator that runs in a single container for AWS service. LocalStack supports a growing number of AWS services.

Bogus - For creating fake data Verify - Snapshot testing for .NET MELT - For testing ILogger usage Stryker - Mutation Testing for .NET TestContainers - run docker programmatically in integration tests

.NET Shop using SQL Server here, but I think something similar to what we do can apply to any stack. We use TestContainers [1] to spin up a container with SQL Server engine running on it. Then use FluentMigrator [2] to provision tables and test data to run XUnit integration tests against. This has worked remarkably well.

[1] https://dotnet.testcontainers.org/