loki
falco
Our great sponsors
loki | falco | |
---|---|---|
79 | 42 | |
21,863 | 6,818 | |
3.4% | 2.7% | |
9.9 | 9.8 | |
5 days ago | 6 days ago | |
Go | C++ | |
GNU Affero General Public License v3.0 | Apache License 2.0 |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
loki
-
List of your reverse proxied services
I also needed to make a small patch to Promtail to make this work: https://github.com/grafana/loki/pull/10256
-
About reading logs
We don't pull logs, we forward logs to a centralized logging service.
-
loki VS openobserve - a user suggested alternative
2 projects | 30 Aug 2023
-
Logs monitoring with Loki, Node.js and Fastify.js
Over the past few months, I've been spending a lot of time creating dashboards on Grafana using Loki for MyUnisoft (the company I work for).
-
OpenObserve: Open source Elasticsearch alternative in Rust for logs. 140x lower storage cost
For log systems you generally don't migrate data. Logs lose value over time. What you want to do is to go ahead and start ingesting data into the new system (OpenObserve in this case) and slowly, the data in the old system will become stale and then you can retire it. However if you need to export logs anyhow, there is no straightforward way in loki to do this. You could run a script to query loki and export it to a file. If found this thread with a sample script - https://github.com/grafana/loki/issues/409
-
i need to visualize all logs from remote dir
Loki
-
I can't recommend serious use of an all-in-one local Grafana Loki setup
If you use Loki as the default logging driver with Docker and the Loki container shuts down, the rest of your containers will freeze up. This has been an issue for almost 3 years.
I installed promtail a few weeks back and I ran into this bug, that has been outstanding for months: https://github.com/grafana/loki/issues/8663 (e.g. a fix had been written but had not been released):
Due to a buffering issue, Loki would exit in case of configuration error without printing any error message or anything at all
There is definitely something weird about how the project is run.
Loki docs are here: https://github.com/grafana/loki/tree/main/docs/sources they are OSS.
The last contributor to the docs was an hour ago (at time of writing this comment) and came from a maintainer not employed by Grafana Labs.
Looking down the recent commits I see lots of activities from non-Grafana employees that have been accepted.
If there are specific issues with contributing docs or code please do point me towards them.
-
Tool to scrape (semi)-structured log files (e.g. log4j)
There are also log forwarding tools like promtail and fluentbit that can be used to both ship logs to something like Loki and produce metrics.
falco
-
Cisco Acquires Splunk
https://github.com/falcosecurity/falco
Like snort, but looks at system calls.
-
An Overview of Kubernetes Security Projects at KubeCon Europe 2023
Falco is a well-known open source security solution originally created by Sysdig. It’s a CNCF incubating project and one of the few (as far as I can tell) options on this list that uses eBPF to scan for vulnerabilities.
-
K8s secret management
Use some kind of SIEM or Falco to alert you to threats (you can't stop them, but a human can always intervene)
-
How to Deploy and Scale Strapi on a Kubernetes Cluster 2/2
Falco, is a security project that can help you detect threats from within your cluster.
-
Go based eBPF projects
https://falco.org/ is a security-focused monitoring and alerting with an eBPF option
- eBPF – Running sandboxed programs in a privileged context such as OS kernel
-
Implement DevSecOps to Secure your CI/CD pipeline
Falco is a cloud native Kubernetes threat detection tool. It can detect unexpected behavior, intrusions, and data theft in real time. In the backend, it uses Linux eBPF technology to trace your system and applications at runtime. For example, it can detect if someone tries to read a secret file inside a container, access a pod as a root user, etc, and trigger a webhook or send logs to the monitoring system. There are similar tools like Tetragon, KubeArmor, and Tracee which also provide Kubernetes runtime security.
-
Blackhat 2022 recap – Trends and highlights
Log everything but know the limits: the golden rule of security is to log everything, but in some cases we exceed the limits. Attackers use this to hide their actions and go unnoticed. At this point, we want to highlight another option. Try to detect at runtime or at the time when these logs occur to avoid the large amount of logs (only one window is sufficient if the initial compromise attack is detected). That is what Falco open source tries to do.
- Live Packet Capture to Grafana
-
Manage Falco easier with Giant Swarm App Platform
Falco is the de facto Kubernetes threat detection engine, and also extends its reach to cloud and Linux hosts. It monitors the behavior of every process in the node and can alert us when something fishy happens.
What are some alternatives?
ClickHouse - ClickHouse® is a free analytics DBMS for big data
fluent-bit - Fast and Lightweight Logs and Metrics processor for Linux, BSD, OSX and Windows
Zabbix - Real-time monitoring of IT components and services, such as networks, servers, VMs, applications and the cloud.
trivy - Find vulnerabilities, misconfigurations, secrets, SBOM in containers, Kubernetes, code repositories, clouds and more
VictoriaMetrics - VictoriaMetrics: fast, cost-effective monitoring solution and time series database
Kyverno - Kubernetes Native Policy Management
ElastiFlow - Network flow analytics (Netflow, sFlow and IPFIX) with the Elastic Stack
loki-multi-tenant-proxy - Grafana Loki multi-tenant Proxy. Needed to deploy Grafana Loki in a multi-tenant way
oauth2-proxy - A reverse proxy that provides authentication with Google, Azure, OpenID Connect and many more identity providers.
OSQuery - SQL powered operating system instrumentation, monitoring, and analytics.
cortex - A horizontally scalable, highly available, multi-tenant, long term Prometheus.
mtail - extract internal monitoring data from application logs for collection in a timeseries database