Top 7 log-parsing Open-Source Projects

scrapydweb

6 3,004 3.6 Python

Web app for Scrapyd cluster management, Scrapy log analysis & visualization, Auto packaging, Timer tasks, Monitor & Alert, and Mobile UI. DEMO :point_right:
loghub

5 1,530 5.3

A large collection of system log datasets for AI-driven log analytics [ISSRE'23]
InfluxDB

www.influxdata.com featured

Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
logparser

2 1,433 7.5 Python

A machine learning toolkit for log parsing [ICSE'19, DSN'16]

Project mention: Log2row: A tool that detects, extracts templates, and structures logs | news.ycombinator.com | 2023-10-06

You use GPT-4 to extract log patterns, does it really need LLM? There are more traditional approach such as https://github.com/logpai/logparser

stanza

8 177 6.6 Go

Fast and lightweight log transport and processing. (by observIQ)
SquadJS

1 149 8.4 JavaScript

Squad Server Script Framework
zero-log-parser

2 29 0.0 Python

Decode Zero log files from the mobile application into text files

Project mention: Zero Motorcycles steps up and makes its service manuals free | news.ycombinator.com | 2023-08-12

This is great news, and somewhat unfortunate something so benign requires celebration.
I used to ride a Zero SR. It was a phenomenal bike but also completely unserviceable in Australia. I ended up reverse engineering the log protocol [1] to get a sense of how it was performing and maintain what I could myself. It was horribly painful, inefficient and completely avoidable spend of time.
Hackability for devices is one thing but when you straddle an energy source that propels you, a fragile meat sack, at speed imo it's essential to have absolute confidence in how the machine behaves.
[1]: https://github.com/zero-motorcycle-community/zero-log-parser

LogParser

1 2 2.7 Python

A Log Parser, that create structured data from log files. (by ZigRazor)
SaaSHub

www.saashub.com featured

SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives

NOTE: The open source projects on this list are ordered by number of github stars. The number of mentions indicates repo mentiontions in the last 12 Months or since we started tracking (Dec 2020).

log-parsing related posts

Zero Motorcycles steps up and makes its service manuals free

1 project | news.ycombinator.com | 12 Aug 2023
Security+ 601 - Where to learn READING LOGS?

1 project | /r/CompTIA | 26 Feb 2023
Any dataset that could be interesting to analyze using text network analysis?

1 project | /r/datasets | 24 Jun 2022
Fluentd vs Promtail with Loki

2 projects | /r/kubernetes | 16 Nov 2021
Syslog Server Recommendations

1 project | /r/sysadmin | 23 Aug 2021
Logforwarding with syslog - namespace included

2 projects | /r/openshift | 23 Jun 2021
Recommendations for syslog servers with web GUI wanted

1 project | /r/sysadmin | 15 Jun 2021
A note from our sponsor - InfluxDB
www.influxdata.com | 7 May 2024

Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality. Learn more →