examples
fluent-bit
examples | fluent-bit | |
---|---|---|
16 | 35 | |
64 | 5,366 | |
- | 1.3% | |
2.6 | 9.8 | |
4 months ago | about 10 hours ago | |
Ruby | C | |
Apache License 2.0 | Apache License 2.0 |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
examples
-
Observability at KubeCon + CloudNativeCon Europe 2024 in Paris
Honeycomb
-
Tracing: Structured Logging, but better in every way
I haven't used anything else, but I'll gladly shill for https://honeycomb.io.
-
Keeping up with my cat's š© using a RaspberryPi
With all of this in place I went a step further and added Opentelemetry to track the stats of how often the routine was being triggered on Honeycomb.
-
Anyone having say 1PB of MySQL data? What efficient storage solution are you using.
Events can be used in many meaningful ways. The Event subsystem of B is pretty much a co-evolution of what honeycomb.io offers, but implemented completely differently - it is on bare-metal, and hence a lot cheaper. Because of that, B never subsampled, but always kept a full low of all events anywhere, no exceptions.
-
āPeople used to take me seriously. Then I became a software vendorā
It should be noted that this is a very oblique ad for http://honeycomb.io. That in no way impugns the content of the post, and in fact, it's given the content of the post that I feel compelled to point out that, ultimately, this is an ad. Because what is sales and advertising, anyway? It's just a way to get you to buy a product, and you can't do that if you've never even heard about the product. I'm not currently in the market for an observability solution (something something splunk) but if I were, I've now heard of them.
The question though, is why does money ruin everything? The naive questions of an open source zealot to a proprietary software salesperson are one thing, but since we all need money to live, why does money being part of the equation (eg if someone was getting paid to post here) ruin things? Would a "donate to open source" button be more successful if the donate text is "buy me a coffee", "buy me a beer", or "upgrade my beer from Coors light"?
https://xkcd.com/2347/ was and is true, and if we don't figure out a way to change that, I don't really see a future for open source.
-
Does anyone else use scatterplots of events?
Very cool to see honeycomb.io is doing that. I'm about to embark on my distributed tracing learning journey, this makes me want to try honeycomb right away.
-
Is there a beginners guide to adding observability to your applications?
Caveat: I work for a vendor in the O11y space (https://honeycomb.io) as a Developer Advocate, however, this advice is generic, not specific to our platform.
-
KubeCon North America 2022: A Retrospective
I spent Day 2 at the Colony Club to attend OTel Unplugged. This event was sponsored by Lightstep, Honeycomb, New Relic, Splunk, Dynatrace, Crowdstrike, and NGINX. I came into the event not knowing what to expect. I can sometimes clamp up when Iām around folks that I donāt know, but because I was helping with the event check-in, I got to say hello to a number of the attendees, which helped break the ice. And it turns out that there were a lot of names that I recognized from my work in the OTel community, and it was nice to connect in person with folks whom Iād only previously met through Slack or Zoom.
- The four pillars of data observability: metrics, metadata, lineage, and logs
-
Honeycomb, Python, and I: an OpenTelemetry Horror Story (With a Happy Ending)
It's no surprise that my apps are mostly written using Sanic as I'm pretty involved with the project. I've been wanting to start testing honeycomb out as well, so it seemed the perfect opportunity to try out.
fluent-bit
-
Observability at KubeCon + CloudNativeCon Europe 2024 in Paris
Fluentbit
-
Fluent Bit with ECS: Configuration Tips and Tricks
$ docker run --rm fluent-bit-dummy WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested Fluent Bit v1.9.10 * Copyright (C) 2015-2022 The Fluent Bit Authors * Fluent Bit is a CNCF sub-project under the umbrella of Fluentd * https://fluentbit.io [2023/12/24 16:06:59] [ info] [fluent bit] version=1.9.10, commit=557c8336e7, pid=1 [2023/12/24 16:06:59] [ info] [storage] version=1.4.0, type=memory-only, sync=normal, checksum=disabled, max_chunks_up=128 [2023/12/24 16:06:59] [ info] [cmetrics] version=0.3.7 [2023/12/24 16:06:59] [ info] [output:stdout:stdout.0] worker #0 started [2023/12/24 16:06:59] [ info] [sp] stream processor started [0] dummy.0: [1703434019.553880465, {"message"=>"custom dummy"}] [0] dummy.0: [1703434020.555768799, {"message"=>"custom dummy"}] [0] dummy.0: [1703434021.550525174, {"message"=>"custom dummy"}] [0] dummy.0: [1703434022.551563050, {"message"=>"custom dummy"}] [0] dummy.0: [1703434023.551944509, {"message"=>"custom dummy"}] [0] dummy.0: [1703434024.550027843, {"message"=>"custom dummy"}] [0] dummy.0: [1703434025.550901801, {"message"=>"custom dummy"}] [0] dummy.0: [1703434026.549279385, {"message"=>"custom dummy"}] ^C[2023/12/24 16:07:08] [engine] caught signal (SIGINT) [0] dummy.0: [1703434027.549678344, {"message"=>"custom dummy"}] [2023/12/24 16:07:08] [ warn] [engine] service will shutdown in max 5 seconds [2023/12/24 16:07:08] [ info] [engine] service has stopped (0 pending tasks) [2023/12/24 16:07:08] [ info] [output:stdout:stdout.0] thread worker #0 stopping... [2023/12/24 16:07:08] [ info] [output:stdout:stdout.0] thread worker #0 stopped
-
Should You Be Scared of Unix Signals?
> Libc is a lot more tricky about signals, since not all libc functions can be safely called from handlers.
And this is a huge thing. People do all kinds of operations in signal handlers completely oblivious to the pitfalls. Pitfalls which often do not manifest, making it a great "it works for me" territory.
I once raised a ticket on fluentbit[1] about it but they have abused signal handlers so thoroughly that I do not think they can mitigate the issue without a major rewriting of the signal and crash handling.
[1] https://github.com/fluent/fluent-bit/issues/4836
-
Vector: a Rust-based lightweight alternative to Fluentd/Logstash
Fluentbit is Fluentd's lightweight alternative to itself.
https://fluentbit.io
- FLaNK Stack Weekly for 14 Aug 2023
-
Ultimate EKS Baseline Cluster: Part 1 - Provision EKS
From here, we can explore other developments and tutorials on Kubernetes, such as o11y or observability (PLG, ELK, ELF, TICK, Jaeger, Pyroscope), service mesh (Linkerd, Istio, NSM, Consul Connect, Cillium), and progressive delivery (ArgoCD, FluxCD, Spinnaker).
-
Fluentbit Kubernetes - How to extract fields from existing logs
From this (https://github.com/fluent/fluent-bit/issues/723), I can see there is no grok support for fluent-bit.
-
Parsing multiline logs using a custom Fluent Bit configuration
apiVersion: v1 kind: ConfigMap metadata: name: fluent-bit-config namespace: newrelic labels: k8s-app: newrelic-logging data: # Configuration files: server, input, filters and output # ====================================================== fluent-bit.conf: | [SERVICE] Flush 1 Log_Level ${LOG_LEVEL} Daemon off Parsers_File parsers.conf HTTP_Server On HTTP_Listen 0.0.0.0 HTTP_Port 2020 @INCLUDE input-kubernetes.conf @INCLUDE output-newrelic.conf @INCLUDE filter-kubernetes.conf input-kubernetes.conf: | [INPUT] Name tail Tag kube.* Path ${PATH} Parser ${LOG_PARSER} DB /var/log/flb_kube.db Mem_Buf_Limit 7MB Skip_Long_Lines On Refresh_Interval 10 filter-kubernetes.conf: | [FILTER] Name multiline Match * multiline.parser multiline-regex [FILTER] Name record_modifier Match * Record cluster_name ${CLUSTER_NAME} [FILTER] Name kubernetes Match kube.* Kube_URL https://kubernetes.default.svc.cluster.local:443 Merge_Log Off output-newrelic.conf: | [OUTPUT] Name newrelic Match * licenseKey ${LICENSE_KEY} endpoint ${ENDPOINT} parsers.conf: | # Relevant parsers retrieved from: https://github.com/fluent/fluent-bit/blob/master/conf/parsers.conf [PARSER] Name docker Format json Time_Key time Time_Format %Y-%m-%dT%H:%M:%S.%L Time_Keep On [PARSER] Name cri Format regex Regex ^(?[^ ]+) (?stdout|stderr) (?[^ ]*) (?.*)$ Time_Key time Time_Format %Y-%m-%dT%H:%M:%S.%L%z [MULTILINE_PARSER] name multiline-regex key_content message type regex flush_timeout 1000 # # Regex rules for multiline parsing # --------------------------------- # # configuration hints: # # - first state always has the name: start_state # - every field in the rule must be inside double quotes # # rules | state name | regex pattern | next state # ------|---------------|--------------------------------|----------- rule "start_state" "/(Dec \d+ \d+\:\d+\:\d+)(.*)/" "cont" rule "cont" "/^\s+at.*/" "cont"
-
Tool to scrape (semi)-structured log files (e.g. log4j)
There are also log forwarding tools like promtail and fluentbit that can be used to both ship logs to something like Loki and produce metrics.
-
How to Deploy and Scale Strapi on a Kubernetes Cluster 2/2
FluentBit, is a logging processor that can help you to push all of your application logs to a central location like an ElasticSearch or OpenSearch cluster.
What are some alternatives?
metriql - The metrics layer for your data. Join us at https://metriql.com/slack
loki - Like Prometheus, but for logs.
otel-cli - OpenTelemetry command-line tool for sending events from shell scripts & similar environments
rsyslog - a Rocket-fast SYStem for LOG processing
nx-go - š Nx plugin to use Go in a Nx Workspace
syslog-ng - syslog-ng is an enhanced log daemon, supporting a wide range of input and output methods: syslog, unstructured text, queueing, SQL & NoSQL.
nxpansion
jaeger - CNCF Jaeger, a Distributed Tracing Platform
keptn - Cloud-native application life-cycle orchestration. Keptn automates your SLO-driven multi-stage delivery and operations & remediation of your applications.
winston - A logger for just about everything.
hyperdx - Resolve production issues, fast. An open source observability platform unifying session replays, logs, metrics, traces and errors powered by Clickhouse and OpenTelemetry.
Grafana - The open and composable observability and data visualization platform. Visualize metrics, logs, and traces from multiple sources like Prometheus, Loki, Elasticsearch, InfluxDB, Postgres and many more.