loki
Logflare
Our great sponsors
loki | Logflare | |
---|---|---|
80 | 11 | |
22,213 | 777 | |
4.0% | 2.6% | |
9.9 | 9.8 | |
2 days ago | 4 days ago | |
Go | Elixir | |
GNU Affero General Public License v3.0 | Apache License 2.0 |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
loki
- Loki 3.0 Released
-
List of your reverse proxied services
I also needed to make a small patch to Promtail to make this work: https://github.com/grafana/loki/pull/10256
-
About reading logs
We don't pull logs, we forward logs to a centralized logging service.
-
loki VS openobserve - a user suggested alternative
2 projects | 30 Aug 2023
-
Logs monitoring with Loki, Node.js and Fastify.js
Over the past few months, I've been spending a lot of time creating dashboards on Grafana using Loki for MyUnisoft (the company I work for).
-
OpenObserve: Open source Elasticsearch alternative in Rust for logs. 140x lower storage cost
For log systems you generally don't migrate data. Logs lose value over time. What you want to do is to go ahead and start ingesting data into the new system (OpenObserve in this case) and slowly, the data in the old system will become stale and then you can retire it. However if you need to export logs anyhow, there is no straightforward way in loki to do this. You could run a script to query loki and export it to a file. If found this thread with a sample script - https://github.com/grafana/loki/issues/409
-
Config files of snaps?
That snap is woefully out of date. The upstream repo was recently updated to 2.8.2, but the snap stable channel has 2.4.1 from 18 months ago. https://github.com/grafana/loki/releases/tag/v2.8.2
-
i need to visualize all logs from remote dir
Loki
- Loki Helm charts that use DynamoDB
-
I can't recommend serious use of an all-in-one local Grafana Loki setup
I installed promtail a few weeks back and I ran into this bug, that has been outstanding for months: https://github.com/grafana/loki/issues/8663 (e.g. a fix had been written but had not been released):
Due to a buffering issue, Loki would exit in case of configuration error without printing any error message or anything at all
There is definitely something weird about how the project is run.
Logflare
-
PostgreSQL Is Enough
btw recently cleaned up my wal cache busting code quite a bit if you're interested.
https://github.com/Logflare/logflare/blob/main/lib/logflare/...
Need to make a lib out of this!!
-
Migrating from Supabase
hey hn, supabase ceo here
the Val Town team were kind enough to share this article with me before they released it. Perhaps you know from previous HN threads that we take customer feedback very seriously. Hearing feedback like this is hard. Clearly the team at Val Town wanted Supabase to be great and we didn’t meet their expectations. For me personally, that hurts. A few quick comments
1. Modifying the database in production: I’ve published a doc on Maturity Models[0]. Hopefully this makes it clear that developers should be using Migrations once their project is live (not using the Dashboard to modify their database live). It also highlights the options for managing dev/local environments. This is just a start. We’re building Preview Databases into the native workflow so that developers don’t need to think about this.
2. Designing for Supabase: Our goal is to make all of Postgres easy, not obligatory. I’ve added a paragraph[1] in the first page in our Docs highlighting that it’s not always a good idea to go all-in on Postgres. We’ll add examples to our docs with “traditional” approaches like Node + Supabase, Rails + Supabase, etc. There are a lot of companies using this approach already, but our docs are overly focused on “the Supabase way” of doing things. There shouldn’t be a reason to switch from Supabase to any other Postgres provider if you want “plain Postgres”.
3. That said, we also want to continue making “all of Postgres” easy to use. We’re committed to building an amazing CLI experience. Like any tech, we’re going to need a few iterations. W’re building tooling for debugging and observability. We have index advisors coming[2]. We recently added Open Telemetry to Logflare[3] and added logging for local development[4]. We’re making platform usage incredibly clear[5]. We aim to make your database indestructible - we care about resilience as much as experience and we’ll make sure we highlight that in future product announcements.
I’ll finish with something that I think we did well: migrating away from Supabase was easy for Val Town, because it’s just Postgres. This is one of our core principles, “everything is portable” (https://supabase.com/docs/guides/getting-started/architectur...). Portability forces us compete on experience. We aim to be the best Postgres hosting service in the world, and we’ll continue to focus on that goal even if we’re not there yet.
[0] Maturity models: https://supabase.com/docs/guides/platform/maturity-model
[1] Choose your comfort level: https://supabase.com/docs/guides/getting-started/architectur...
[2] Index advisor: https://database.dev/olirice/index_advisor
[3] Open Telemetry: https://github.com/Logflare/logflare/pull/1466
[4] Local logging: https://supabase.com/blog/supabase-logs-self-hosted
[5] Usage: https://twitter.com/kiwicopple/status/1658683758718124032?s=...
-
How to get access logs from Cloudflare?
https://logflare.app/ is awesome, pipes into BiqQuery and is really easy to use and WAY cheaper than logpush. Depending on the amount of traffic, it's only a few dollars a month.
-
Supabase Logs: open-source logging server
hey hn, supabase ceo here
this one is a long-time coming and it's a continuation of our acquisition of Logflare[0]. Since the acquisition we've be open-source-ing the server, which you can find here: https://github.com/Logflare/logflare
Logflare handles about 1.5 billion log-events everyday on supabase. It's built with Elixir and has no problems with that workload.
This is really just the start of the Logflare updates. All logs are currently ingested into BigQuery, and we are adding support for Clickhouse and other OLAP backends. Over time this will function very much like an open source Sentry alternative, where you can ingest data from various sources.
The team will be around if you have any questions about the technical implementation
[0] acquision: https://supabase.com/blog/supabase-acquires-logflare
-
Supabase Logs: open source logging server
Logflare was available under a BSL license prior to joining Supabase. We’ve since changed the license to Apache 2.0, aligning it with our open source philosophy.
-
Logging requests with cloudflare
https://www.cloudflare.com/apps/logflare https://logflare.app/
-
Tools for Querying Logs with SQL
Logflare, now a part of Supabase, aims to streamline the logging experience for Cloudflare-, Elixir-, and Vercel-based applications. However, it can be adapted to support any type of log. Logflare provides structured logging ability without limits or added latency. It aims to provide the best performance with minimal overhead when processing logs for supported application platforms.
What are some alternatives?
ClickHouse - ClickHouse® is a free analytics DBMS for big data
Hydra - Open source API gateway with integrated cache and data transformations.
fluent-bit - Fast and Lightweight Logs and Metrics processor for Linux, BSD, OSX and Windows
n2o - ⭕ N2O: Distributed WebSocket Application Server ISO 20922
Zabbix - Real-time monitoring of IT components and services, such as networks, servers, VMs, applications and the cloud.
Phoenix Toggl - Toggl tribute done with Elixir, Phoenix Framework, React and Redux.
VictoriaMetrics - VictoriaMetrics: fast, cost-effective monitoring solution and time series database
ExChat - (Not maintaining) A Slack-like app by Elixir, Phoenix & React(redux)
ElastiFlow - Network flow analytics (Netflow, sFlow and IPFIX) with the Elastic Stack
Phoenix Battleship - The Good Old game, built with Elixir, Phoenix, React and Redux
loki-multi-tenant-proxy - Grafana Loki multi-tenant Proxy. Needed to deploy Grafana Loki in a multi-tenant way
majremind