Top 3 C Stream Processing Projects

fluent-bit

13 4,581 9.9 C

Fast and Lightweight Logs and Metrics processor for Linux, BSD, OSX and Windows

Project mention: Parsing multiline logs using a custom Fluent Bit configuration | dev.to | 2023-05-25

apiVersion: v1 kind: ConfigMap metadata: name: fluent-bit-config namespace: newrelic labels: k8s-app: newrelic-logging data: # Configuration files: server, input, filters and output # ====================================================== fluent-bit.conf: | [SERVICE] Flush 1 Log_Level ${LOG_LEVEL} Daemon off Parsers_File parsers.conf HTTP_Server On HTTP_Listen 0.0.0.0 HTTP_Port 2020 @INCLUDE input-kubernetes.conf @INCLUDE output-newrelic.conf @INCLUDE filter-kubernetes.conf input-kubernetes.conf: | [INPUT] Name tail Tag kube.* Path ${PATH} Parser ${LOG_PARSER} DB /var/log/flb_kube.db Mem_Buf_Limit 7MB Skip_Long_Lines On Refresh_Interval 10 filter-kubernetes.conf: | [FILTER] Name multiline Match * multiline.parser multiline-regex [FILTER] Name record_modifier Match * Record cluster_name ${CLUSTER_NAME} [FILTER] Name kubernetes Match kube.* Kube_URL https://kubernetes.default.svc.cluster.local:443 Merge_Log Off output-newrelic.conf: | [OUTPUT] Name newrelic Match * licenseKey ${LICENSE_KEY} endpoint ${ENDPOINT} parsers.conf: | # Relevant parsers retrieved from: https://github.com/fluent/fluent-bit/blob/master/conf/parsers.conf [PARSER] Name docker Format json Time_Key time Time_Format %Y-%m-%dT%H:%M:%S.%L Time_Keep On [PARSER] Name cri Format regex Regex ^(?[^ ]+) (?stdout|stderr) (?[^ ]*) (?.*)$ Time_Key time Time_Format %Y-%m-%dT%H:%M:%S.%L%z [MULTILINE_PARSER] name multiline-regex key_content message type regex flush_timeout 1000 # # Regex rules for multiline parsing # --------------------------------- # # configuration hints: # # - first state always has the name: start_state # - every field in the rule must be inside double quotes # # rules | state name | regex pattern | next state # ------|---------------|--------------------------------|----------- rule "start_state" "/(Dec \d+ \d+\:\d+\:\d+)(.*)/" "cont" rule "cont" "/^\s+at.*/" "cont"
PipelineDB

1 2,553 0.0 C

High-performance time-series aggregation for PostgreSQL

Project mention: The database inside out with event streams | reddit.com/r/coding | 2023-04-12

I was experimenting with continuous sql queries in postgres for the same purpose. https://github.com/pipelinedb/pipelinedb
CodiumAI

codium.ai
sponsored

TestGPT | Generating meaningful tests for busy devs. Get non-trivial tests (and trivial, too!) suggested right inside your IDE, so you can code smart, create more value, and stay confident when you push.
kurento

1 43 9.7 C

Kurento WebRTC Media Server

Project mention: Google abandoned Fastlane, it has no maintainers currently | news.ycombinator.com | 2023-02-19

Let me add a very relevant anecdote of the difference between those things.
Kurento the project is Apache-2 open-source, you'll find all its source code here: https://github.com/Kurento/kurento
its website was kurento .org but the trademark of the name (and the domain) is owned by Twilio, who recently had the nice gesture of redirecting all accesses from this domain to their own, without prior warning. No replies to emails, or reasons stated. Not that they needed one though, as they are indeed the owners and can do whatever they want. Just a nice "contribution" they made to OSS, it seems.

NOTE: The open source projects on this list are ordered by number of github stars. The number of mentions indicates repo mentiontions in the last 12 Months or since we started tracking (Dec 2020). The latest post mention was on 2023-05-25.