vector
agent
Our great sponsors
vector | agent | |
---|---|---|
95 | 21 | |
16,366 | 1,539 | |
4.8% | 4.4% | |
9.9 | 9.9 | |
7 days ago | 7 days ago | |
Rust | Go | |
Mozilla Public License 2.0 | Apache License 2.0 |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
vector
- FLaNK AI Weekly 18 March 2024
- Vector: A high-performance observability data pipeline
-
Hacks to reduce cloud spend
we are doing something similar with OTEL but we are looking at using https://vector.dev/
-
About reading logs
We don't pull logs, we forward logs to a centralized logging service.
-
Self hosted log paraer
opensearch - amazon fork of Elasticsearch https://opensearch.org/docs/latestif you do this an have distributed log sources you'd use logstash for, bin off logstash and use vector (https://vector.dev/) its better out of the box for SaaS stuff.
-
creating a centralize syslog server with elastic search
I have done something similar in the past: you can send the logs through a centralized syslog servers (I suggest syslog-ng) and from there ingest into ELK. For parsing I am advice to use something like Vector, is a lot more faster than logstash. When you have your logs ingested correctly, you can create your own dashboard in Kibana. If this fit your requirements, no need to install nginx (unless you want to use as reverse proxy for Kibana), php and mysql.
-
Show HN: Homelab Monitoring Setup with Grafana
I think there's nothing currently that combines both logging and metrics into one easy package and visualizes it, but it's also something I would love to have.
Vector[1] would work as the agent, being able to collect both logs and metrics. But the issue would then be storing it. I'm assuming the Elastic Stack might now be able to do both, but it's just to heavy to deal with in a small setup.
A couple of months ago I took a brief look at that when setting up logging for my own homelab (https://pv.wtf/posts/logging-and-the-homelab). Mostly looking at the memory usage to fit it on my synology. Quickwit[2] and Log-Store[3] both come with built in web interfaces that reduce the need for grafana, but neither of them do metrics.
- [1] https://vector.dev
-
Retaining Logs generated by service running in pod.
Log to stdout/stderr and collect your logs with a tool like vector (vector.dev) and send it to something like Grafana Loki.
-
Lightweight logging on RPi?
I would recommend that you run vector as a systems service so you don't have to worry about managing it. Here is a basic config to do that - https://github.com/vectordotdev/vector/blob/master/distribution/systemd/vector.service .
-
Monitoring traefik access logs easily
You could have a look at Grafana Loki, it's easy to run (single binary for a small setup). Shipping your logs can be done by Promtail or something like Vector. They're both lightweight log shippers with support for Loki.
agent
-
Ask HN: Best solution for homelab service monitoring?
And has an agent that can be run on the host to monitor it: https://github.com/grafana/agent
- Grafana to sumologic pricing
-
Best unicorn monitoring system?
For example, there are things like https://github.com/grafana/agent
-
Grafana agent JSON Schema
Hey, I would like to lint my agent config in vscode. Did someone know where I can find the agent config JSON schema ? I've searched on https://github.com/grafana/agent and https://grafana.com/docs/agent/latest/ but cant find anything like this ...
-
Monitoring Internet Quality and Speed
Grafana Agent can stream data back to you.
-
Distributed tracing with Envoy, Kuma, Grafana Agent, and Jaeger
At the time, Jaeger did not support OTEL format, which meant we needed our telemetry collector to export spans in Jaeger format. We were able to push this small fix to add a Jaeger exporter on the Agent.
-
Those of you using prometheus as part of your observability stack, what approach did you take to scaling to scrape 25+ clusters, and why? Is Thanos the answer to my problems?
Furthermore, would recommend Grafana Agent OR Prometheus Agent in this case since you probably don't need the Prometheus UI in each Cluster as well as the Alerting stuff that is inside Prometheus. (Mimir will do the ruling stuff for you). Grafana Agent also has an Operator mode if you want to use ServiceMonitor and PodMonitor CustomResources.
- Promtail on windows as service
-
Prometheus Agent Mode, for Metric Forwarding
There were a bunch of alternatives to solve this before, like grafana agent [0], vm agent [1] or opentelemetry collector [2].
[0] - https://github.com/grafana/agent
[1] - https://github.com/VictoriaMetrics/VictoriaMetrics/tree/mast...
[2] - https://github.com/open-telemetry/opentelemetry-collector
-
Shell script to monitor remote VPS?
This might be a reasonable use case for the Grafana Agent. It's basically a Prometheus instance, but without the local storage. Just forwards the data.
What are some alternatives?
graylog - Free and open log management
loki - Like Prometheus, but for logs.
Fluentd - Fluentd: Unified Logging Layer (project under CNCF)
PushProx - Proxy to allow Prometheus to scrape through NAT etc.
syslog-ng - syslog-ng is an enhanced log daemon, supporting a wide range of input and output methods: syslog, unstructured text, queueing, SQL & NoSQL.
operator - Kubernetes operator for Victoria Metrics
OpenSearch - 🔎 Open source distributed and RESTful search engine.
thanos - Highly available Prometheus setup with long term storage capabilities. A CNCF Incubating project.
tracing - Application level tracing for Rust.
helm-charts - Helm charts for VictoriaMetrics, VictoriaLogs and ecosystem
qryn - qryn is a polyglot, high-performance observability framework for ClickHouse. Ingest, store and analyze logs, metrics and telemetry traces from any agent supporting Loki, Prometheus, OTLP, Tempo, Elastic, InfluxDB and many more formats and query transparently using Grafana or any other compatible client.
snmp_exporter - SNMP Exporter for Prometheus