cortex VS loki

There are many Projects like Thanos, M3, Cortex, and Victoriametrics. But Thanos is the most popular among these. Thanos addresses these issues with Prometheus and is the ideal solution for scaling Prometheus in environments with extensive metrics or multiple clusters where we require a global view of historical metrics. In this blog, we will explore the components of Thanos and will try to simplify its architecture by building it step by step, starting with the main components. We will also have a demo using k6-metrics. Before diving into Thanos, I recommend reading our blog on Prometheus Monitoring with Prometheus if you are not already familiar with Prometheus.

now if its more metric data you are using and want to do APM, prometheus is your man https://prometheus.io/, want to make prometheus your full time job? deploy cortex https://cortexmetrics.io/, honorable mention in the metrics space, Zabbix, https://www.zabbix.com/ I've seen use cases of zabbix going way beyond its intended use its a fantastic tool

Yes, but also no. The Prometheus ecosystem already has two FOSS time-series databases that are complementary to Prometheus itself. Thanos and Mimir. Not to mention M3db, developed at Uber, and Cortex, then ancestor of Mimir. There's a bunch of others I won't mention as it would take too long.

You can use the Remote write feature to send to a centralized location. It would have to be scalable like Cortex https://cortexmetrics.io/

Cortex is not really good for high-cardinality metrics (if you are talking about https://github.com/cortexproject/cortex)

For a homelab I think prometheus + grafana is easy to get started and scales well. There are lots of ways to set up the architecture. Prometheus can write to a directory on a filesystem, it can be set to write to a remote server, and there are other projects to integrate object storage (s3, minio, etc) or influxdb for long term storage and downsampling.

Cortex (it is renamed to Mimir recently).

Disclosure: I work for AWS, but I don't work on the Amazon Managed Service for Prometheus. I have my own very long held opinions about Free and Open Source software, and I am only speaking for myself.

To me, the AGPLv3 license isn't about forcing software users to "give changes back" to a project. It is about giving the permissions to users of software that are necessary for Software Freedom [1] when they access a program over a network. In practice, that means that changes often flow "upstream" to copyleft licensed programs one way or another. But it was never about obligating changes to be "given back" to upstream. In my personal opinion, you should be "free to fork" Free and Open Source Software (FOSS). Indeed, the Grafana folks seem to have decided to do that with Grafana Mimir.

Personally, I hope that they accept contributions under the AGPLv3 license, and hold themselves to the same obligations that others are held to with regard to providing corresponding source code of derivative works when it is made available to users over a network. In my personal opinion, too often companies use a contributor agreement that excuses them from those obligations, and also allows them to sell the software to others under licenses that do not carry copyleft obligations. See [2] for a blog post that goes into some detail about this.

If you look at the Coretex project MAINTAINERS file [3], you will see that there are two folks listed that currently work at AWS, but no other company other than Grafana Labs today. I would love to see more diversity in maintainers for a project like this, as I think too many maintainers from any one company isn't the best for long term project sustainability.

I think if you look at the Cortex Community Meeting minutes [4], you can see that AWS folks are regularly "showing up" in healthy numbers, and working collaboratively with anyone who accepts the open invitation to participate. There have been some pretty big improvements to Coretex that have merged lately, like some of the work on parallel compaction [5, 6].

TL;DR, I think it is easy to jump to some conclusions about how things are going in a FOSS project that don't hold water if you do some cursory exploration. I think best way to know what's going on in a project is to get involved!

--

[1] the rights needed to: run the program for any purpose; to study how the program works, and modify it; to redistribute copies; to distribute copies of modified versions to others

[2] https://meshedinsights.com/2021/06/14/legally-ignoring-the-l...

[3] https://github.com/cortexproject/cortex/blob/master/MAINTAIN...

[4] https://docs.google.com/document/d/1shtXSAqp3t7fiC-9uZcKkq3m...

[5] https://aws.amazon.com/blogs/opensource/scaling-cortex-with-...

[6] https://github.com/cortexproject/cortex/pull/4624

We intend to support a wide variety of destinations. Syslog and Loki are currently under development and are expected to be released in the coming weeks. If you would like your favorite tools to be supported as a destination, vote on this Github discussion!

Loki is a log collation tool from the Grafana family of observability solutions. It aggregates, groups, and labels logs from your applications, helping you troubleshoot problems and monitor activity. Although Loki is a general-purpose tool, it's well-suited to Kubernetes and comes with several Kubernetes-specific features. It automatically scrapes and indexes metadata from your Kubernetes workload objects, such as Pod labels, to accompany your Pod logs.

It's like Prometheus, but for logs. Okay it's not really to do with the Norse or Greek gods, instead Loki is a horizontally-scalable, highly-available, multi-tenant log aggregation system inspired by the open source project Prometheus. Built by Grafana Labs, Loki is designed for ease of use. Instead of indexing the contents of the logs, Loki provides a set of labels for each log stream. The latest update includes query acceleration with Bloom filters, native OTel support, Helm charts, and more. Check out the changelog for all the major changes and deprecations.

I also needed to make a small patch to Promtail to make this work: https://github.com/grafana/loki/pull/10256

We don't pull logs, we forward logs to a centralized logging service.

Over the past few months, I've been spending a lot of time creating dashboards on Grafana using Loki for MyUnisoft (the company I work for).

For log systems you generally don't migrate data. Logs lose value over time. What you want to do is to go ahead and start ingesting data into the new system (OpenObserve in this case) and slowly, the data in the old system will become stale and then you can retire it. However if you need to export logs anyhow, there is no straightforward way in loki to do this. You could run a script to query loki and export it to a file. If found this thread with a sample script - https://github.com/grafana/loki/issues/409