Logs

Open-source projects categorized as Logs
Edit details
Language: + Go + Rust + TypeScript + C + PHP + C++ + Java + Python
Topics: Logging Observability Monitoring Metrics Logger
SaaSHub - Software Alternatives and Reviews
SaaSHub helps you find the best software and product alternatives
www.saashub.com
Sponsored

Top 23 Log Open-Source Projects

  • signoz

    SigNoz is an open-source observability platform native to OpenTelemetry with logs, traces and metrics in a single application. An open-source alternative to DataDog, NewRelic, etc. 🔥 🖥. 👉 Open source Application Performance Monitoring (APM) & Observability tool

    • Project mention: Show HN: OneUptime – open-source Datadog Alternative | news.ycombinator.com | 2024-04-02

    You should also check out SigNoz [1], we are an open-core alternative to DataDog - based natively on OpenTelemetry. We also have a cloud product if you don't want to host yourself

    [1] https://signoz.io

  • vector

    A high-performance observability data pipeline.

    • Project mention: Docker Log Observability: Analyzing Container Logs in HashiCorp Nomad with Vector, Loki, and Grafana | dev.to | 2024-04-19

    job "vector" { datacenters = ["dc1"] # system job, runs on all nodes type = "system" group "vector" { count = 1 network { port "api" { to = 8686 } } ephemeral_disk { size = 500 sticky = true } task "vector" { driver = "docker" config { image = "timberio/vector:0.30.0-debian" ports = ["api"] volumes = ["/var/run/docker.sock:/var/run/docker.sock"] } env { VECTOR_CONFIG = "local/vector.toml" VECTOR_REQUIRE_HEALTHY = "false" } resources { cpu = 100 # 100 MHz memory = 100 # 100MB } # template with Vector's configuration template { destination = "local/vector.toml" change_mode = "signal" change_signal = "SIGHUP" # overriding the delimiters to [[ ]] to avoid conflicts with Vector's native templating, which also uses {{ }} left_delimiter = "[[" right_delimiter = "]]" data=<

  • InfluxDB

    Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

    InfluxDB logo

  • openobserve

    🚀 10x easier, 🚀 140x lower storage cost, 🚀 high performance, 🚀 petabyte scale - Elasticsearch/Splunk/Datadog alternative for 🚀 (logs, metrics, traces, RUM, Error tracking, Session replay).

    • Project mention: Show HN: OneUptime – open-source Datadog Alternative | news.ycombinator.com | 2024-04-02

    Lot of interesting OSS observability products coming out in recent years. One of the more impressive(and curious for many reasons) IMHO is OpenObserve: https://github.com/openobserve/openobserve .

    As opposed to just a stack, they are implementing just about the whole backend shebang from scratch.

  • Benthos

    Fancy stream processing made operationally mundane

    • Project mention: Ask HN: Who is hiring? (December 2023) | news.ycombinator.com | 2023-12-01

  • hyperdx

    Resolve production issues, fast. An open source observability platform unifying session replays, logs, metrics, traces and errors powered by Clickhouse and OpenTelemetry.

    • Project mention: Show HN: HyperDX Local – Open-source Datadog alternative for local debugging/dev | news.ycombinator.com | 2024-04-02

    Hi HN, Mike and Warren here! We’ve been building an open source local-dev-friendly mode for HyperDX (hyperdx.io). It's a single Docker container that lets you view logs, metrics, and traces for local development in a similar way you can use them for production (live tail, filter/correlate logs & spans, build charts, flamegraph, etc.) Basically, imagine you can run Datadog in a single container for local development. We does this by spinning up an OpenTelemetry collector, Clickhouse DB and HyperDX UI all in a single container - it only takes a few seconds to start and lets you start live tailing your local logs and traces immediately (and graph metrics of course).

    README (w/ demo gif): https://github.com/hyperdxio/hyperdx/blob/main/LOCAL.md

    We started building local mode as it became one of our most loved [1] community issue after sharing HyperDX. It made sense - as we constantly use HyperDX to debug our own issues locally, and have found it to be a huge productivity boost for things like…

    1. Tailing multiple container logs (and grepping/isolating them) without a ton of different terminal splits open at the same time.

    2. Be able to sensibly view structured logs and correlated traces to debug local issues (because you’re using structured logging and tracing right?) instead of adding ad-hoc console statements that print out exactly what I could’ve gotten from tracing.

    3. Testing telemetry are actually emitting/correlating as expected, before shipping it all the way out to prod and realizing I accidentally created a very expensive high-cardinality metric or a span that has the wrong properties.

    We spent some time packaging & tuning our existing OSS stack to run in a single container with less memory/space requirements by staring at `dive` to slim down the image and applied incantations from documentation until memory usage improved. Additionally we removed a few non-local-friendly things like authentication requirements and extraneous services.

    It still has all the goodness you’d want - so you can full text search your logs/traces, live tail all your events, view spans correlated with logs (and vice versa), create dashboards based on logs, metrics, traces, and is fully OpenTelemetry compatible - just point your Otel SDK/collector to http://localhost:4318 (or 4317 for the grpc folks) and you’re already good to go.

    I’m excited to share what we’ve been working on and would love to hear your feedback and opinions!

    Spin up the container yourself to try it out:

    docker run -p 8000:8000 -p 4318:4318 -p 4317:4317 -p 8080:8080 -p 8002:8002 hyperdx/hyperdx-local

    Main Open Source Repo: https://github.com/hyperdxio/hyperdx

    Hosted Demo (in case you want to play around in a cloud sandbox instead): https://api.hyperdx.io/login/demo

    HyperDX Landing Page: https://hyperdx.io

    [1]: https://github.com/hyperdxio/hyperdx/issues/7

  • quickwit

    Cloud-native search engine for observability. An open-source alternative to Datadog, Elasticsearch, Loki, and Tempo.

    • Project mention: Show HN: Search on S3 Using AWS Lambda | news.ycombinator.com | 2024-01-21

  • fluent-bit

    Fast and Lightweight Logs and Metrics processor for Linux, BSD, OSX and Windows

    • Project mention: Observability at KubeCon + CloudNativeCon Europe 2024 in Paris | dev.to | 2024-03-26

    Fluentbit

  • WorkOS

    The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.

    WorkOS logo

  • mtail

    extract internal monitoring data from application logs for collection in a timeseries database

    • Project mention: i need to visualize all logs from remote dir | /r/sysadmin | 2023-05-19

    You can do that with something like mtail. Basically write expressions that match your logs and produce metrics.

  • log-viewer

    Fast and beautiful Log Viewer for Laravel

  • odigos

    Distributed tracing without code changes. 🚀 Instantly monitor any application using OpenTelemetry and eBPF

    • Project mention: Setup odigos in Ubuntu | dev.to | 2024-04-10

    Welcome 👋 to this blog. Did you want to use odigos on your Ubuntu machine but don't know how to start? Then this blog will definitely help you and it also helps you to understand the basic aspect of the odigos project.

  • uptrace

    Open source APM: OpenTelemetry traces, metrics, and logs

    • Project mention: Show HN: Traces, metrics, and logs using OpenTelemetry and ClickHouse | news.ycombinator.com | 2024-03-27

  • stern

    ⎈ Multi pod and container log tailing for Kubernetes -- Friendly fork of https://github.com/wercker/stern (by stern)

    • Project mention: ☸️ Kubernetes: From your docker-compose file to a cluster with Kompose | dev.to | 2024-03-09

    deploy: stage: deploy image: alpine/k8s:1.29.1 variables: NAMESPACE: $CI_COMMIT_REF_SLUG before_script: # init namespace - kubectl config use-context $KUBE_CONTEXT - kubectl create namespace $NAMESPACE || true # download tools - curl --show-error --silent --location https://github.com/stern/stern/releases/download/v1.22.0/stern_1.22.0_linux_amd64.tar.gz | tar zx --directory /usr/bin/ stern && chmod 755 /usr/bin/stern && stern --version - curl --show-error --silent --location https://github.com/kubernetes/kompose/releases/download/v1.32.0/kompose-linux-amd64 -o /usr/local/bin/kompose && chmod a+x /usr/local/bin/kompose && kompose version # show logs asynchronously. Timeout to avoid hanging indefinitely when an error occurs in script section - timeout 1200 stern -n $NAMESPACE "app-" --tail=0 --color=always & # in background, tail new logs if any (current and incoming) pod with this regex as name - timeout 1200 kubectl -n $NAMESPACE get events --watch-only & # in background, tail new events in background script: # first delete CrashLoopBackOff pods, polluting logs - kubectl -n $NAMESPACE delete pod `kubectl -n $NAMESPACE get pods --selector app.kubernetes.io/component=$MODULE | awk '$3 == "CrashLoopBackOff" {print $1}'` || true # now deploying - kompose convert --out k8s/ - kubectl apply -n $NAMESPACE -f k8s/ - echo -e "\e[93;1mWaiting for the new app version to be fully operational...\e[0m" # waiting for successful deployment - kubectl -n $NAMESPACE rollout status deploy/app-db - kubectl -n $NAMESPACE rollout status deploy/app-back - kubectl -n $NAMESPACE rollout status deploy/app-front # on any error before this line, the script will still wait for these threads to complete, so the initial timeout is important. Adding these commands to after_script does not help - pkill stern || true - pkill kubectl || true after_script: # show namespace content - kubectl config use-context $KUBE_CONTEXT - kubectl -n $NAMESPACE get deploy,service,ingress,pod

  • chainsaw

    Rapidly Search and Hunt through Windows Forensic Artefacts

    • Project mention: Agent event queue is flooded. Check the agent configuration | /r/Wazuh | 2023-06-30

  • rita

    Real Intelligence Threat Analytics (RITA) is a framework for detecting command and control communication through network traffic analysis.

  • klogg

    Really fast log explorer based on glogg project

    • Project mention: Toolong: Terminal application to view, tail, merge, and search log files | news.ycombinator.com | 2024-02-09

    I'd love to see a tool that lets you modify large files efficiently.

    I had to replace line 4 of a 200 GB SQL dump, it took a substantial amount of compute time to perform a find / replace with sed and it also required having over double the disk space since sed creates a temp file before it writes out the new file.

    Using a hex editor could have worked but it seemed too risky because data integrity was really important.

    There's also some other scenarios where maybe you have a massive file and the tool that's using it (such as a SQL import) throws an error on line 1,025,421. Trying to find what the contents of that line is on the command line could be time consuming if you need to read in the whole file. For read operations I know there's a few graphical tools like https://github.com/variar/klogg that efficiently let you scan, search and jump to points in a file quickly but I haven't found a good one on the command line.

  • hayabusa

    Hayabusa (隼) is a sigma-based threat hunting and fast forensics timeline generator for Windows event logs.

    • Project mention: Hayabusa: Sigma-based forensics timeline generator for Windows event logs | news.ycombinator.com | 2024-04-24

  • kail

    kubernetes log viewer

  • Logbook

    An extensible Java library for HTTP request and response logging

    • Project mention: Spring Library To Potentially Log and Store Every API request? | /r/java | 2023-07-10

    If by API the author means HTTP, Logbook satifies most of the requirements.

  • loghub

    A large collection of system log datasets for AI-driven log analytics [ISSRE'23]

  • saw

    Fast, multi-purpose tool for AWS CloudWatch Logs

    • Project mention: Lambda recursive loop detection - but not (yet) for S3 | dev.to | 2023-07-17

    We use saw to tail lambda log:

  • WebTools.bundle

    WebTools is a collection of tools for Plex Media Server. Like the Unsupported AppStore (UAS)

    • Project mention: Help Please!!! | /r/PleX | 2023-04-26

    The old WebTools isn't something you open in Plex itself, you have to go to http://[your-pms-server-ip-address]:33400. That said, what exactly are you trying to use it for? Chances are there are other ways to do it that don't involve using an unmaintained plugin.

  • Awesome-Cybersecurity-Datasets

    A curated list of amazingly awesome Cybersecurity datasets

  • SigLens

    100x Efficient Log Management than Splunk, Reduce your observability cost by 90%

    • Project mention: Siglens: A columnar OLAP db that can execute queries 50x faster than ClickHouse | news.ycombinator.com | 2024-02-20

    I found their claims exaggerated: https://github.com/siglens/siglens/issues/created_by/alexey-...

  • SaaSHub

    SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives

    SaaSHub logo
NOTE: The open source projects on this list are ordered by number of github stars. The number of mentions indicates repo mentiontions in the last 12 Months or since we started tracking (Dec 2020).

Logs related posts

Index

What are some of the best open-source Log projects? This list will help you:

Project Stars
1 signoz 16,886
2 vector 16,427
3 openobserve 9,368
4 Benthos 7,559
5 hyperdx 6,082
6 quickwit 6,052
7 fluent-bit 5,321
8 mtail 3,740
9 log-viewer 3,262
10 odigos 3,020
11 uptrace 2,892
12 stern 2,786
13 chainsaw 2,541
14 rita 2,462
15 klogg 1,992
16 hayabusa 1,922
17 kail 1,918
18 Logbook 1,714
19 loghub 1,518
20 saw 1,387
21 WebTools.bundle 1,290
22 Awesome-Cybersecurity-Datasets 1,171
23 SigLens 995

Sponsored
SaaSHub - Software Alternatives and Reviews
SaaSHub helps you find the best software and product alternatives
www.saashub.com