k3sup VS fluent-bit

# Maintainer: Talha Altinel pkgname=k3sup pkgver=0.13.0 pkgrel=1 pkgdesc='A tool to bootstrap K3s over SSH in < 60s' arch=('x86_64') url='https://github.com/alexellis/k3sup' license=('MIT') depends=('glibc' 'openssh') makedepends=('git' 'go>=1.20') source=("${pkgname}-${pkgver}.tar.gz::https://github.com/alexellis/k3sup/archive/${pkgver}.tar.gz") sha256sums=('24939844ac6de581eb05ef6425c89c32b2d0e22800f1344c19b2164eec846c92') _commit=('1d2e443ea56a355cc6bd0a14a8f8a2661a72f2e8') build() { cd "$pkgname-$pkgver" export CGO_CPPFLAGS="${CPPFLAGS}" export CGO_CFLAGS="${CFLAGS}" export CGO_CXXFLAGS="${CXXFLAGS}" export CGO_LDFLAGS="${LDFLAGS}" export GOFLAGS="-buildmode=pie -trimpath -mod=readonly -modcacherw" go build \ -ldflags "-s -w -X github.com/alexellis/k3sup/cmd.Version=$pkgver -X github.com/alexellis/k3sup/cmd.GitCommit=$_commit" \ -o k3sup \ . for shell in bash fish zsh; do ./k3sup completion "$shell" > "$shell-completion" done } package() { cd "$pkgname-$pkgver" install -Dm755 -t "$pkgdir/usr/bin" k3sup mkdir -p "${pkgdir}/usr/share/bash-completion/completions/" mkdir -p "${pkgdir}/usr/share/zsh/site-functions/" mkdir -p "${pkgdir}/usr/share/fish/vendor_completions.d/" install -Dm644 bash-completion "$pkgdir/usr/share/bash-completion/completions/k3sup" install -Dm644 fish-completion "$pkgdir/usr/share/fish/vendor_completions.d/k3sup.fish" install -Dm644 zsh-completion "$pkgdir/usr/share/zsh/site-functions/_k3sup" install -Dm644 -t "$pkgdir/usr/share/licenses/$pkgname" LICENSE }

I recommend learning docker first, then pick a vps host from vpsbenchmarks, then use k3sup to deploy a kubernetes cluster on that, then follow a getting-started kubernetes tutorial from there. You'll also want to buy a domain name with tld-list and then provision a TLS certificate with cert-manager and letsencrypt (skip steps 1-4 because Google Cloud is overpriced).

I just installed k3s yesterday using k3sup on 6 VMs (3 masters, 3 workers) each with 2GB RAM ( limited by the actual RAM on hardware, for now ) with Ubuntu 22.04 as the base OS.

k3s installed with k3sup, longhorn for storage, kube-vip for API VIP, and MetalLB for service load balancer using local subnet, and of course Rancher.

My applause to Alex Ellis for writing a clear, direct call to arms!

Their work is super useful and interesting. I've added them to my list of sponsorships: https://github.com/sponsors/alexellis

Yeah, this is the answer, but I would use this with K3S: https://github.com/alexellis/k3sup

$ curl -sLS https://get.k3sup.dev | sh x86_64 Downloading package https://github.com/alexellis/k3sup/releases/download/0.12.12/k3sup as /home/ec2-user/k3sup Download complete. ============================================================ The script was run as a user who is unable to write to /usr/local/bin. To complete the installation the following commands may need to be run manually. ============================================================ sudo cp k3sup /usr/local/bin/k3sup ================================================================ alexellis's work on k3sup needs your support https://github.com/sponsors/alexellis ================================================================ No nos devolverá nada, pero podremos correr lo siguiente para saber si k3sup efectivamente se instalo:

Fluentbit

$ docker run --rm fluent-bit-dummy WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested Fluent Bit v1.9.10 * Copyright (C) 2015-2022 The Fluent Bit Authors * Fluent Bit is a CNCF sub-project under the umbrella of Fluentd * https://fluentbit.io [2023/12/24 16:06:59] [ info] [fluent bit] version=1.9.10, commit=557c8336e7, pid=1 [2023/12/24 16:06:59] [ info] [storage] version=1.4.0, type=memory-only, sync=normal, checksum=disabled, max_chunks_up=128 [2023/12/24 16:06:59] [ info] [cmetrics] version=0.3.7 [2023/12/24 16:06:59] [ info] [output:stdout:stdout.0] worker #0 started [2023/12/24 16:06:59] [ info] [sp] stream processor started [0] dummy.0: [1703434019.553880465, {"message"=>"custom dummy"}] [0] dummy.0: [1703434020.555768799, {"message"=>"custom dummy"}] [0] dummy.0: [1703434021.550525174, {"message"=>"custom dummy"}] [0] dummy.0: [1703434022.551563050, {"message"=>"custom dummy"}] [0] dummy.0: [1703434023.551944509, {"message"=>"custom dummy"}] [0] dummy.0: [1703434024.550027843, {"message"=>"custom dummy"}] [0] dummy.0: [1703434025.550901801, {"message"=>"custom dummy"}] [0] dummy.0: [1703434026.549279385, {"message"=>"custom dummy"}] ^C[2023/12/24 16:07:08] [engine] caught signal (SIGINT) [0] dummy.0: [1703434027.549678344, {"message"=>"custom dummy"}] [2023/12/24 16:07:08] [ warn] [engine] service will shutdown in max 5 seconds [2023/12/24 16:07:08] [ info] [engine] service has stopped (0 pending tasks) [2023/12/24 16:07:08] [ info] [output:stdout:stdout.0] thread worker #0 stopping... [2023/12/24 16:07:08] [ info] [output:stdout:stdout.0] thread worker #0 stopped

> Libc is a lot more tricky about signals, since not all libc functions can be safely called from handlers.

And this is a huge thing. People do all kinds of operations in signal handlers completely oblivious to the pitfalls. Pitfalls which often do not manifest, making it a great "it works for me" territory.

I once raised a ticket on fluentbit[1] about it but they have abused signal handlers so thoroughly that I do not think they can mitigate the issue without a major rewriting of the signal and crash handling.

[1] https://github.com/fluent/fluent-bit/issues/4836

Fluentbit is Fluentd's lightweight alternative to itself.

https://fluentbit.io

From here, we can explore other developments and tutorials on Kubernetes, such as o11y or observability (PLG, ELK, ELF, TICK, Jaeger, Pyroscope), service mesh (Linkerd, Istio, NSM, Consul Connect, Cillium), and progressive delivery (ArgoCD, FluxCD, Spinnaker).

From this (https://github.com/fluent/fluent-bit/issues/723), I can see there is no grok support for fluent-bit.

apiVersion: v1 kind: ConfigMap metadata: name: fluent-bit-config namespace: newrelic labels: k8s-app: newrelic-logging data: # Configuration files: server, input, filters and output # ====================================================== fluent-bit.conf: | [SERVICE] Flush 1 Log_Level ${LOG_LEVEL} Daemon off Parsers_File parsers.conf HTTP_Server On HTTP_Listen 0.0.0.0 HTTP_Port 2020 @INCLUDE input-kubernetes.conf @INCLUDE output-newrelic.conf @INCLUDE filter-kubernetes.conf input-kubernetes.conf: | [INPUT] Name tail Tag kube.* Path ${PATH} Parser ${LOG_PARSER} DB /var/log/flb_kube.db Mem_Buf_Limit 7MB Skip_Long_Lines On Refresh_Interval 10 filter-kubernetes.conf: | [FILTER] Name multiline Match * multiline.parser multiline-regex [FILTER] Name record_modifier Match * Record cluster_name ${CLUSTER_NAME} [FILTER] Name kubernetes Match kube.* Kube_URL https://kubernetes.default.svc.cluster.local:443 Merge_Log Off output-newrelic.conf: | [OUTPUT] Name newrelic Match * licenseKey ${LICENSE_KEY} endpoint ${ENDPOINT} parsers.conf: | # Relevant parsers retrieved from: https://github.com/fluent/fluent-bit/blob/master/conf/parsers.conf [PARSER] Name docker Format json Time_Key time Time_Format %Y-%m-%dT%H:%M:%S.%L Time_Keep On [PARSER] Name cri Format regex Regex ^(?[^ ]+) (?stdout|stderr) (?[^ ]*) (?.*)$ Time_Key time Time_Format %Y-%m-%dT%H:%M:%S.%L%z [MULTILINE_PARSER] name multiline-regex key_content message type regex flush_timeout 1000 # # Regex rules for multiline parsing # --------------------------------- # # configuration hints: # # - first state always has the name: start_state # - every field in the rule must be inside double quotes # # rules | state name | regex pattern | next state # ------|---------------|--------------------------------|----------- rule "start_state" "/(Dec \d+ \d+\:\d+\:\d+)(.*)/" "cont" rule "cont" "/^\s+at.*/" "cont"

There are also log forwarding tools like promtail and fluentbit that can be used to both ship logs to something like Loki and produce metrics.

FluentBit, is a logging processor that can help you to push all of your application logs to a central location like an ElasticSearch or OpenSearch cluster.