busybox-w32 VS dagster

Even in environments such as win32, we have https://frippery.org/busybox/ that is just fucking awesome. Staying the size below an 1mb while being extremely fast. Unlike the shitty python package which has 40mb archive size and leave breadcrumbs for me to cleanup all over my filesystem.

Win32 port is here: https://frippery.org/busybox/

Look into busybox for windows https://frippery.org/busybox/. Pretty bad ass even with it’s downsides of missing applets and such

Vim by itself means no supporting unix environment. It's useful to call out to powerful external tools not present by default on Windows. I fill that gap with busybox-w32. It's not a big deal once solved.

Sure, it's not necessary, but a few simple, nice tools (<600kiB for an entire suite of extended unix utilities) makes thing a whole lot simpler on a platform devoid of nice tools.

To operate on many files you'll need better tools than what Windows gives you. One option is busybox-w32 (important caveat: doesn't support unicode paths), which will get you some basic command line tools. For example, to gzip compress every file under the current directory, including subdirectories (leaving the originals behind with -k):

busybox-w32 includes a cal applet. If that's all you care about, you can just rename busybox.exe to cal.exe.

busybox-w32: standard unix utilities for Windows. It's a BusyBox port.

IHMO, even better is to just use POSIX sh in your Makefile and simply make it a build requirement. It's easy to obtain a reasonable sh even on Windows (Cygwin, MSYS2, busybox-w32), and to further support exactly this I include sh alongside make in my development kit distribution. This uniformity lets me hit all operating systems with the same Makefile. I use EXE from the environment to determine the binary file extension, if any.

My recommendation is to continue on with the tutorial, then look at one of the larger example projects especially the ones named “project_”, and you should understand most of it. Of what you don't understand and you're curious about, look into the relevant concept page for the functions in the docs.

I found this example that helped me - https://github.com/dagster-io/dagster/tree/master/examples/project_fully_featured/project_fully_featured

I would recommend the following: - https://www.mage.ai/ - https://dagster.io/ - https://www.prefect.io/ - https://metaflow.org/ - https://zenml.io/home

In Helm terms: there are 2 charts, namely the system: dagster/dagster (values.yaml), and the user code: dagster/dagster-user-deployments (values.yaml). Note that you have to set dagster-user-deployments.enabled: true in the dagster/dagster values-yaml to enable this.

Dagster seemed really cool when I looked into it as an alternative to airflow. I especially like the software defined assets and built-in lineage which I haven't seen in any other tool. However it seems it does not support RBAC which is a pretty big issue if you want a self-service type of architecture, see https://github.com/dagster-io/dagster/issues/2219. It does seem like it's available in their hosted version, but I wanted to run it myself on k8s.

Dagster? https://dagster.io

One that I haven't seen on here yet: dagster