dbt-unit-testing VS pg-mem

If you use dbt for transformations Dbt Unit Testing (https://github.com/EqualExperts/dbt-unit-testing) is getting some attention (https://www.thoughtworks.com/radar/languages-and-frameworks?blipid=202304042)

dbt-unit-testing

We use this and take an example-based tests approach for any non-trivial tables: https://github.com/EqualExperts/dbt-unit-testing

> How do you test some SQL logic in isolation?

I do this using sql

1. Extracting an 'ephemeral model' to different model file

2. Mock out this model in upstream model in unit tests https://github.com/EqualExperts/dbt-unit-testing

3. Write unit tests for this model.

This is not different than regular software development in a language like java.

I would argue its even better better because unit tests are always in tabular format and pretty easy to understand. Java unit tests on other hand are never read by devs in practice.

I haven't done it yet but there are some popular blogs as well as a DBT package someone created.

> I really don’t understand the communities obsession with unwieldy tools like DBT.

It lets me write test first sql transforms. I never thought TDD sql would be possible. My sql is so much more readable with common logic extracted into ephmeral models. I practice same method to write clear code to write sql, eg: too many mocks = refactor into separate model ( class) .

I think DBT made this possible with refs that can be swapped out with mocks. This is the awesome library I am using https://github.com/EqualExperts/dbt-unit-testing

I've used pgmem https://github.com/oguimbal/pg-mem for the last couple of years for the same thing.

I was wondering the other day how to classify tests that use a test double like pg-mem, which isn't a mock but isn't the Dockerized test DB either :

https://github.com/oguimbal/pg-mem

In my case, I use TypeORM with PostgreSQL, and there's pg-mem to run an instance in memory, it supports most of the common functionality of PostgreSQL but you will need to do some adjustment to your code to be within the limits.

Postgres in the browser is possible through pg-mem: "pg-mem is an experimental in-memory emulation of a postgres database" but it also suffers from no persistence. If you can persist to a file somewhere then read it in on startup (and if your local data isn't huge) this might work.

Dont get me wrong: I am trying to contribute to opensource as well, so I get that supporting small projects can be demanding. There's nothing wrong in not spending your weekends on OS. But not asking for help, nor specifying that a project is unmaintained, nor even answering issues & pull requests for years feels just wrong.

A pure Javascript in memory emulation of Posgres, to help writing better node tests https://github.com/oguimbal/pg-mem

Okay, I had a bit of spare time,I've implemented that, and it is now available with [email protected]

To get a glimpse of what I'm talking about, you can clone this repo and follow "Development" instructions (by the way this is a small OS lib I maintain, I wrote about it here)