vector
core
Our great sponsors
vector | core | |
---|---|---|
96 | 241 | |
16,427 | 2,965 | |
5.2% | 4.3% | |
9.9 | 9.9 | |
6 days ago | 4 days ago | |
Rust | PHP | |
Mozilla Public License 2.0 | BSD 2-clause "Simplified" License |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
vector
-
Docker Log Observability: Analyzing Container Logs in HashiCorp Nomad with Vector, Loki, and Grafana
job "vector" { datacenters = ["dc1"] # system job, runs on all nodes type = "system" group "vector" { count = 1 network { port "api" { to = 8686 } } ephemeral_disk { size = 500 sticky = true } task "vector" { driver = "docker" config { image = "timberio/vector:0.30.0-debian" ports = ["api"] volumes = ["/var/run/docker.sock:/var/run/docker.sock"] } env { VECTOR_CONFIG = "local/vector.toml" VECTOR_REQUIRE_HEALTHY = "false" } resources { cpu = 100 # 100 MHz memory = 100 # 100MB } # template with Vector's configuration template { destination = "local/vector.toml" change_mode = "signal" change_signal = "SIGHUP" # overriding the delimiters to [[ ]] to avoid conflicts with Vector's native templating, which also uses {{ }} left_delimiter = "[[" right_delimiter = "]]" data=<
- FLaNK AI Weekly 18 March 2024
- Vector: A high-performance observability data pipeline
-
Hacks to reduce cloud spend
we are doing something similar with OTEL but we are looking at using https://vector.dev/
-
About reading logs
We don't pull logs, we forward logs to a centralized logging service.
-
Self hosted log paraer
opensearch - amazon fork of Elasticsearch https://opensearch.org/docs/latestif you do this an have distributed log sources you'd use logstash for, bin off logstash and use vector (https://vector.dev/) its better out of the box for SaaS stuff.
-
creating a centralize syslog server with elastic search
I have done something similar in the past: you can send the logs through a centralized syslog servers (I suggest syslog-ng) and from there ingest into ELK. For parsing I am advice to use something like Vector, is a lot more faster than logstash. When you have your logs ingested correctly, you can create your own dashboard in Kibana. If this fit your requirements, no need to install nginx (unless you want to use as reverse proxy for Kibana), php and mysql.
-
Show HN: Homelab Monitoring Setup with Grafana
I think there's nothing currently that combines both logging and metrics into one easy package and visualizes it, but it's also something I would love to have.
Vector[1] would work as the agent, being able to collect both logs and metrics. But the issue would then be storing it. I'm assuming the Elastic Stack might now be able to do both, but it's just to heavy to deal with in a small setup.
A couple of months ago I took a brief look at that when setting up logging for my own homelab (https://pv.wtf/posts/logging-and-the-homelab). Mostly looking at the memory usage to fit it on my synology. Quickwit[2] and Log-Store[3] both come with built in web interfaces that reduce the need for grafana, but neither of them do metrics.
- [1] https://vector.dev
-
Retaining Logs generated by service running in pod.
Log to stdout/stderr and collect your logs with a tool like vector (vector.dev) and send it to something like Grafana Loki.
-
Lightweight logging on RPi?
I would recommend that you run vector as a systems service so you don't have to worry about managing it. Here is a basic config to do that - https://github.com/vectordotdev/vector/blob/master/distribution/systemd/vector.service .
core
-
Dnsmasq wins the first BlueHats Prize
dnsmasq can be used for wildcard domain aliases in OPNsense firewall, https://github.com/opnsense/core/issues/4145#issuecomment-12...
- OPNsense Wildcard Support in Firewall Host Alias (2022)
-
How to bypass public IP and NAT
Firmware's like Asuswrt-Merlin or OpenWRT can support dynamic-dns, or you can do like I do and run something like OPNsense in an x86 VM with a NIC passed through, or buy an inexpensive firewall appliance (up to 500mbps/1gbps/10gbps).
-
Stop ISP from getting into my Router
The easiest solution is to buy your own router, set it up, disable the router functionality on the Fritzbox 7590 and plug your router into it. It'll be cheaper and easier than a Cisco Firewall, but if you want to go the dedicated firewall route then I would recommenced OPNsense
-
Near Constant PTR lookups from localhost
Much searching lead me to a possible patch Unbound.inc for how it was handling aliases for 23.7 -> https://github.com/opnsense/core/pull/5925
-
The world in which IPv6 was a good design
[2]: https://github.com/opnsense/core/issues/2544
-
OpenBSD Innovations
BSDs may not have a significant presence on desktops, but they're well known in the networking world for their reliability. They also were the foundation used to build OSes for specific applications. OpnSense and XigmaNAS, for example, are two excellent FreeBSD based applications aimed at firewalling/security and NAS/services.
https://opnsense.org/
https://xigmanas.com/xnaswp/
- Root user access denied when adding new users?
- OPNsense: Open-source security platform
What are some alternatives?
graylog - Free and open log management
docker-pihole-unbound - Run Pi-Hole + Unbound on Docker
Fluentd - Fluentd: Unified Logging Layer (project under CNCF)
openwrt - Linux distribution for embedded devices
agent - Vendor-neutral programmable observability pipelines.
openNDS - openNDS (open Network Demarcation Service) is a high performance, small footprint, Captive Portal. It provides a border control gateway between a public local area network and the Internet.
syslog-ng - syslog-ng is an enhanced log daemon, supporting a wide range of input and output methods: syslog, unstructured text, queueing, SQL & NoSQL.
asuswrt-merlin.ng - Third party firmware for Asus routers (newer codebase)
OpenSearch - 🔎 Open source distributed and RESTful search engine.
UTMFW - UTM Firewall on OpenBSD
tracing - Application level tracing for Rust.
unifios-utilities - A collection of enhancements for UnifiOS based devices