plumber VS argo-cd

Streamdal

4. Go to the provided UI (or run the CLI app) and be able to peek into what your app is reading or writing, like with `tail -f`.

And that's basically it. There's a bunch more functionality in the project but we find this to be the most immediately useful part. Every developer we've shown this to has said "I wish I had this at my gig at $company" - and we feel exactly the same. We are devs and this is what we’ve always wanted, hundreds of times - a way to just quickly look at the data our software is producing in real-time, without having to jump through any hoops.

If you want to learn more about the "why" and the origin of this project - you can read about it here: https://streamdal.com/manifesto

— — —

HOW DOES IT WORK?

The SDK establishes a long-running session with the server (using gRPC) and "listens" for commands that are forwarded to it all the way from the UI -> server -> SDK.

The commands are things like: "show me the data that you are currently consuming", "apply these rules to all data that you produce", "inspect the schema for all data", and so on.

The SDK interprets the command and either executes Wasm-based rules against the data it's processing or if it's a `tail` request - it'll send the data to the server, which will forward it to the UI for display.

The SDK IS part of the critical path but it does not have a dependency on the server. If the server is gone, you won't be able to use the UI or send commands to the SDKs, but that's about it - the SDKs will continue to work and attempt to reconnect to the server behind the scenes.

— — —

TECHNICAL BITS

The project consists of a lot of "buzzwordy" tech: we use gRPC, grpc-Web, protobuf, redis, Wasm, Deno, ReactFlow, and probably a few other things.

The server is written in Go, all of the Wasm is Rust and the UI is Typescript. There are SDKs for Go, Python, and Node. We chose these languages for the SDKs because we've been working in them daily for the past 10+ years.

The reasons for the tech choices are explained in detail here: https://docs.streamdal.com/en/resources-support/open-source/

— — —

LAST PART

OK, that's it. What do you think? Is it useful? Can we answer anything?

- If you like what you're seeing, give our repo a star: https://github.com/streamdal/streamdal

Checkout plumber https://github.com/streamdal/plumber if you want all the Postgres changes sent to basically any type of broker queue https://docs.streamdal.com/en/data-ingestion/relay/postgresql-cdc/. I would say Nat's Jetstream is probably the way to go if you have K8s running already. It's a dead simple service written in Go. Just make sure you allocate enough memory to Jetstream.

I recommend Streamdal. The connecting agent is open source and distributed by default, so it will scale horizontally WAY better than Debezium. All data ingested is indexed into parquet as well, and you can do serverless functions/transforms on the platform to reduce Snowflake compute costs.

For CDC and event driven part of the stack, Plumber is a great free tool. That project is going to be adding sampling soon too - this can def help with the cost of ETL.

https://github.com/batchcorp/plumber check it out if you want to get into event driven systems

That’s awesome to see. My coworker and I always figured Go would be perfect for this. Going to be a serious amount of work! I see you use NATS as well. Big fan of it. Checkout our project https://github.com/batchcorp/plumber if you end up needing to inspect or send messages while deving against it.

Checkout plumber https://github.com/batchcorp/plumber join our slack https://launchpass.com/streamdal we got a pretty knowledgeable group

The code above will create the argocd Kubernetes namespace and deploy the latest stable manifest. If you would like to install a specific manifest, have a look here.

In this blog post, we will demonstrate how easy and fast it is to deploy Sveltos on an RKE2 cluster with the help of ArgoCD, register two RKE2 Cluster API (CAPI) clusters and create a ClusterProfile to deploy Prometheus and Grafana Helm charts down the managed CAPI clusters.

Argo CD

$version = (Invoke-RestMethod https://api.github.com/repos/argoproj/argo-cd/releases/latest).tag_name Invoke-WebRequest -Uri "https://github.com/argoproj/argo-cd/releases/download/$version/argocd-windows-amd64.exe" -OutFile "argocd.exe"

This is cool - I can think of some projects that are amazing as first contributors, and others I can think of that are terrible.

One thing I think the tool doesn't address is why someone should contribute to a particular project. Having stars is interesting, and a proxy for at least historical activity, but also kind of useless here - take argoproj/argo-cd [1] as an example - 14.5k stars, with a backlog of 2.7k issues and an issue tracker that's a real mess.

Either way, I think this tool is neat for trying to gain some experience in a project purely based on language.

[1] https://github.com/argoproj/argo-cd/issues?q=is%3Aissue+is%3...

In our case, our team went ahead with Solution B, as that was the only solution present when the issue occurred. However, with the release of Argo CD 2.8.0 (released on August 7, 2023), things have changed - for the better :). Now, there are two ways to handle the sharding issue with the Argo CD Application Controller:

$ kubectl create namespace argocd //Next, let's apply the yaml configuration files for ArgoCd $ kubectl apply -n argocd -f https://raw.githubusercontent.com/argoproj/argo-cd/stable/manifests/install.yaml //Now we can view the pods created in the ArgoCD namespace. $ kubectl get pods -n argocd //To interact with the API Server we need to deploy the CLI: $ curl --silent --location -o /usr/local/bin/argocd https://github.com/argoproj/argo-cd/releases/download/v2.4.7/argocd-linux-amd64 $ chmod +x /usr/local/bin/argocd //Expose argocd-server $ kubectl patch svc argocd-server -n argocd -p '{"spec": {"type": "LoadBalancer"}}' //Wait about 2 minutes for the LoadBalancer creation $ kubectl get svc -n argocd //Get pasword and decode it. $ kubectl get secret argocd-initial-admin-secret -n argocd -o yaml $ echo WXVpLUg2LWxoWjRkSHFmSA== | base64 --decode

From here, we can explore other developments and tutorials on Kubernetes, such as o11y or observability (PLG, ELK, ELF, TICK, Jaeger, Pyroscope), service mesh (Linkerd, Istio, NSM, Consul Connect, Cillium), and progressive delivery (ArgoCD, FluxCD, Spinnaker).

lol! Wham! Third choice! https://github.com/argoproj/argo-cd

If you mean for each app, I don't think it's listed anywhere though you may find it in `repo-server` logs. Like so