dbt-unit-testing
distrobox
dbt-unit-testing | distrobox | |
---|---|---|
7 | 402 | |
404 | 8,976 | |
1.5% | - | |
7.7 | 9.6 | |
16 days ago | 3 days ago | |
Shell | Shell | |
MIT License | GNU General Public License v3.0 only |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
dbt-unit-testing
-
The SQL Unit Testing Landscape: 2023
If you use dbt for transformations Dbt Unit Testing (https://github.com/EqualExperts/dbt-unit-testing) is getting some attention (https://www.thoughtworks.com/radar/languages-and-frameworks?blipid=202304042)
-
Data-eng related highlights from the latest Thoughtworks Tech Radar
dbt-unit-testing
- I'm not getting it...what's the point of DBT?
-
Ask HN: How do you test SQL?
We use this and take an example-based tests approach for any non-trivial tables: https://github.com/EqualExperts/dbt-unit-testing
-
SQL should be your default choice for data engineering pipelines
> How do you test some SQL logic in isolation?
I do this using sql
1. Extracting an 'ephemeral model' to different model file
2. Mock out this model in upstream model in unit tests https://github.com/EqualExperts/dbt-unit-testing
3. Write unit tests for this model.
This is not different than regular software development in a language like java.
I would argue its even better better because unit tests are always in tabular format and pretty easy to understand. Java unit tests on other hand are never read by devs in practice.
-
Unit testing with dbt
I haven't done it yet but there are some popular blogs as well as a DBT package someone created.
-
Modern Data Modeling: Start with the End?
> I really don’t understand the communities obsession with unwieldy tools like DBT.
It lets me write test first sql transforms. I never thought TDD sql would be possible. My sql is so much more readable with common logic extracted into ephmeral models. I practice same method to write clear code to write sql, eg: too many mocks = refactor into separate model ( class) .
I think DBT made this possible with refs that can be swapped out with mocks. This is the awesome library I am using https://github.com/EqualExperts/dbt-unit-testing
distrobox
-
Windows 11 now comes with its own adware
Regarding the stability issue on a dev machine - you may be interested in playing with one of the immutable-os distros, such as SilverBlue (fedora based).
The high-level take-away is you can't break your actual OS since it's root filesystem is read-only, and you use "pet" containers (on docker, podman, whatever) to do your work in. Applications are either sandboxed via Flatpak, or installed/run inside your pet containers. If your pet container dies, you cry about it for a moment, and when you're ready you get a new one - your actual os and other containers remain unaffected.
I use distrobox[1] to create/run the pet containers.
[1] https://github.com/89luca89/distrobox
-
Tools for Linux Distro Hoppers
Distrobox is a tool that enables us to try Linux distro CLI, including their package manager. This requires a containerization tool (e.g., Docker). In Windows, this can be achieved using WSL (Windows Subsystem for Linux)
- Distrobox: Use any Linux distribution inside your terminal
-
Fedora Atomic Desktops
I use containerized versions of things, ubuntu and chainguard images mostly.
You can always create containers with init if that's how you want to do that though. Some distros publish images that come that way: https://github.com/89luca89/distrobox/blob/main/docs/useful_...
-
Raspberry Pi is manufacturing 70K Raspberry Pi 5s per week
https://news.ycombinator.com/item?id=38505448 ... https://github.com/89luca89/distrobox/blob/main/docs/useful_...
-
Operating System?
Yes, you can do that but I've seen others use something like distrobox to run linux inside of SteamOS: https://github.com/89luca89/distrobox/blob/main/docs/posts/steamdeck_guide.md
-
How much will I screw up my system after installing Merkuro Calendar (KDE Akonadi application), formerly called Kalendar, on GNOME?
For such cases you might use something like this: https://github.com/89luca89/distrobox
-
Battery consumption of using remote development with WSL2?
Btw #3: Depending on what the user is trying to accomplish, e.g. maybe to make WSL(2) itself more of a "subsystem" than a "container engine", using something like Distrobox or nsbox.dev can be a good idea (along with Docker or Podman in Distrobox's case; the other one uses systemd-nspawn).
-
Cannot run containers with Distrobox
1. Find here in "Containers Distros" section the distro image that you want to install ("Toolbox" versions are better because they are configured for Distrobox) and get it URL: https://distrobox.it/compatibility/#containers-distros 2. Use that URL to create Distrobox: distrobox create -i registry.fedoraproject.org/fedora-toolbox:39 -n fedora_1_39 3. Enter Distrobox fedora_1_39: distrobox enter fedora_1_39 4. You are already in Distrobox console. Look at the name in console, it should be include the container name. 5. To exit Distrobox: exit 6. If you run: distrobox list you will see all distroboxes on the system. You will also see that distrobox that we exited is still running. 7. To stop distrobox use commands: distrobox stop fedora_1_39
- In-depth Distrobox tutorial/ or video?
What are some alternatives?
sqlglot - Python SQL Parser and Transpiler
toolbox - Tool for interactive command line environments on Linux
data-diff - Compare tables within or across databases
wsl-distrod - Distrod is a meta-distro for WSL 2 which installs Ubuntu, Arch, Debian, Gentoo, etc. with systemd in a minute for you. Distrod also has built-in auto-start feature on Windows startup and port forwarding ability.
dbt-expectations - Port(ish) of Great Expectations to dbt test macros
docker-android - Android in docker solution with noVNC supported and video recording
sqlx - 🧰 The Rust SQL Toolkit. An async, pure Rust SQL crate featuring compile-time checked queries without a DSL. Supports PostgreSQL, MySQL, and SQLite.
flatpak - Linux application sandboxing and distribution framework
SS-Unit - A 100% T-SQL based unit testing framework for SQL Server
rustdesk - An open-source remote desktop, and alternative to TeamViewer.
hash-db - Experimental distributed pseudomultimodel keyvalue database (it uses python dictionaries) imitating dynamodb querying with join only SQL support, distributed joins and simple Cypher graph support and document storage
toolbox-vscode - Toolbox Visual Studio Code integration