mirrord VS vector

So, you've been using mirrord to simplify your development process (if you haven’t, go here!). Naturally, you want the traffic from the app you're debugging to go through the cluster environment, so your app can communicate with its clustery pals. There is a problem though: your latest change adds some new columns to the database, and you don’t want to modify the database in the cluster and affect everyone else working on it. You do have a local instance of the database that you can modify, so your app can use that, but you still want it to talk to all the other components in the cluster. So what now? The new outgoing traffic filter feature is here to solve exactly this type of problem!

I had the pleasure of talking to Eyal @ CTO at Metalbear and the maintainer of Mirrord. I got some crazy insights.

If you want to take mirrord for a spin, check out the quick start guide. We’d love to hear about your experience or just general thoughts - chat us up on our Discord or open an issue or discussion on GitHub.

We're building an open-source tool called mirrord which lets you run a local process in the context of a pod in your cloud environment. We often get asked how mirrord is different from Telepresence and so we decided to write a short blog post about it, which we hope would be valuable to those interested in local Kubernetes development:

Hands-on tutorial of mirrord.dev with the creators and Rawkode!

if you are interested in k8s, iptables, hooking libc, asm etc https://github.com/metalbear-co/mirrord

I gave my first CNCF talk in Toronto yesterday, talking about https://github.com/metalbear-co/mirrord , how all the features work, and how it's engineered!

I am thinking about using https://vector.dev/ but would also love opinions on the best deal for lower or reasonable cost storage/querying of logs. Thanks!

job "vector" { datacenters = ["dc1"] # system job, runs on all nodes type = "system" group "vector" { count = 1 network { port "api" { to = 8686 } } ephemeral_disk { size = 500 sticky = true } task "vector" { driver = "docker" config { image = "timberio/vector:0.30.0-debian" ports = ["api"] volumes = ["/var/run/docker.sock:/var/run/docker.sock"] } env { VECTOR_CONFIG = "local/vector.toml" VECTOR_REQUIRE_HEALTHY = "false" } resources { cpu = 100 # 100 MHz memory = 100 # 100MB } # template with Vector's configuration template { destination = "local/vector.toml" change_mode = "signal" change_signal = "SIGHUP" # overriding the delimiters to [[ ]] to avoid conflicts with Vector's native templating, which also uses {{ }} left_delimiter = "[[" right_delimiter = "]]" data=<

we are doing something similar with OTEL but we are looking at using https://vector.dev/

We don't pull logs, we forward logs to a centralized logging service.

opensearch - amazon fork of Elasticsearch https://opensearch.org/docs/latestif you do this an have distributed log sources you'd use logstash for, bin off logstash and use vector (https://vector.dev/) its better out of the box for SaaS stuff.

I have done something similar in the past: you can send the logs through a centralized syslog servers (I suggest syslog-ng) and from there ingest into ELK. For parsing I am advice to use something like Vector, is a lot more faster than logstash. When you have your logs ingested correctly, you can create your own dashboard in Kibana. If this fit your requirements, no need to install nginx (unless you want to use as reverse proxy for Kibana), php and mysql.

I think there's nothing currently that combines both logging and metrics into one easy package and visualizes it, but it's also something I would love to have.

Vector[1] would work as the agent, being able to collect both logs and metrics. But the issue would then be storing it. I'm assuming the Elastic Stack might now be able to do both, but it's just to heavy to deal with in a small setup.

A couple of months ago I took a brief look at that when setting up logging for my own homelab (https://pv.wtf/posts/logging-and-the-homelab). Mostly looking at the memory usage to fit it on my synology. Quickwit[2] and Log-Store[3] both come with built in web interfaces that reduce the need for grafana, but neither of them do metrics.

- [1] https://vector.dev

Log to stdout/stderr and collect your logs with a tool like vector (vector.dev) and send it to something like Grafana Loki.