Making a Homegrown ClickHouse Log for $20/mo

This page summarizes the projects mentioned and recommended in the original post on

Our great sponsors
  • Appwrite - The Open Source Firebase alternative introduces iOS support
  • Klotho - AWS Cloud-aware infrastructure-from-code toolbox [NEW]
  • InfluxDB - Access the most powerful time series database as a service
  • Sonar - Write Clean JavaScript Code. Always.
  • manticoresearch

    Easy to use open source fast database for search | Good alternative to Elasticsearch now | Drop-in replacement for E in the ELK soon

  • qryn

    qryn is a polyglot, high-performance observability framework for ClickHouse. Ingest, store and analyze logs, metrics and telemetry traces from any agent supporting Loki, Prometheus, OTLP, Tempo, Elastic, InfluxDB and many more formats and query transparently using Grafana or any other compatible client.

  • Appwrite

    Appwrite - The Open Source Firebase alternative introduces iOS support . Appwrite is an open source backend server that helps you build native iOS applications much faster with realtime APIs for authentication, databases, files storage, cloud functions and much more!

  • CPython

    The Python programming language

    "The Coroner recorded every instruction executed, with its inputs and results, writing over the oldest record when it filled up. When a program crashed, you could punch out a full record of what it was doing in most of its last second and analyze the problem at your ease. I have often wished that PCs offered such an advanced feature."

    So essentially buffer all logs into an in-memory circular buffer of capacity N. If a log record is emitted that has a certain severity/level; flush all records from the buffer to disk/clickHouse/grafana/whatever.

    The python MemoryHandler[2] almost implements the said technique except that particular handler also flushes when buffer is full; which is not particularly what I would want.



  • signoz

    SigNoz is an open-source APM. It helps developers monitor their applications & troubleshoot problems, an open-source alternative to DataDog, NewRelic, etc. 🔥 🖥. 👉 Open source Application Performance Monitoring (APM) & Observability tool

    This sounds fairly similar to self-hosting SigNoz[1] except they use the OpenTelemetry Collector[2] in the place where the author has a custom Redis queue consumer.



  • log-analytics-starter-kit

    Build your own Log Analytics using Tinybird

    This is awesome! We did something similar for our internal logging at Tinybird (which itself is built on ClickHouse) and I recently turned into it a (very simplified) Starter Kit that others can fork and use in their projects.

    Rather than writing to files and using an agent like beats to tail them, it sends logs directly from the application code with a basic POST request. Obviously, you could just as happily tail the file and forward the logs, but this approach reduced the footprint of tools and made it work in serverless environments.

  • Klotho

    AWS Cloud-aware infrastructure-from-code toolbox [NEW]. Build cloud backends with Infrastructure-from-Code (IfC), a revolutionary technique for generating and updating cloud infrastructure. Try IfC with AWS and Klotho now (Now open-source)

NOTE: The number of mentions on this list indicates mentions on common posts plus user suggested alternatives. Hence, a higher number means a more popular project.

Suggest a related project

Related posts