cortex VS loki

Official Product Page: Cortex Metrics

Cortex is a horizontally scalable, highly available, multi-tenant prometheus alternative.

There are many Projects like Thanos, M3, Cortex, and Victoriametrics. But Thanos is the most popular among these. Thanos addresses these issues with Prometheus and is the ideal solution for scaling Prometheus in environments with extensive metrics or multiple clusters where we require a global view of historical metrics. In this blog, we will explore the components of Thanos and will try to simplify its architecture by building it step by step, starting with the main components. We will also have a demo using k6-metrics. Before diving into Thanos, I recommend reading our blog on Prometheus Monitoring with Prometheus if you are not already familiar with Prometheus.

now if its more metric data you are using and want to do APM, prometheus is your man https://prometheus.io/, want to make prometheus your full time job? deploy cortex https://cortexmetrics.io/, honorable mention in the metrics space, Zabbix, https://www.zabbix.com/ I've seen use cases of zabbix going way beyond its intended use its a fantastic tool

Yes, but also no. The Prometheus ecosystem already has two FOSS time-series databases that are complementary to Prometheus itself. Thanos and Mimir. Not to mention M3db, developed at Uber, and Cortex, then ancestor of Mimir. There's a bunch of others I won't mention as it would take too long.

You can use the Remote write feature to send to a centralized location. It would have to be scalable like Cortex https://cortexmetrics.io/

Cortex is not really good for high-cardinality metrics (if you are talking about https://github.com/cortexproject/cortex)

For a homelab I think prometheus + grafana is easy to get started and scales well. There are lots of ways to set up the architecture. Prometheus can write to a directory on a filesystem, it can be set to write to a remote server, and there are other projects to integrate object storage (s3, minio, etc) or influxdb for long term storage and downsampling.

Cortex (it is renamed to Mimir recently).

GitHub Repository: Grafana Loki GitHub

auth_enabled: false server: http_listen_port: 3100 grpc_listen_port: 9096 log_level: debug grpc_server_max_concurrent_streams: 1000 common: instance_addr: 127.0.0.1 path_prefix: /tmp/loki storage: filesystem: chunks_directory: /tmp/loki/chunks rules_directory: /tmp/loki/rules replication_factor: 1 ring: kvstore: store: inmemory query_range: results_cache: cache: embedded_cache: enabled: true max_size_mb: 100 limits_config: metric_aggregation_enabled: true schema_config: configs: - from: 2020-10-24 store: tsdb object_store: filesystem schema: v13 index: prefix: index_ period: 24h pattern_ingester: enabled: true metric_aggregation: loki_address: localhost:3100 ruler: alertmanager_url: http://localhost:9093 frontend: encoding: protobuf # By default, Loki will send anonymous, but uniquely-identifiable usage and configuration # analytics to Grafana Labs. These statistics are sent to https://stats.grafana.org/ # # Statistics help us better understand how Loki is used, and they show us performance # levels for most users. This helps us prioritize features and documentation. # For more information on what's sent, look at # https://github.com/grafana/loki/blob/main/pkg/analytics/stats.go # Refer to the buildReport method to see what goes into a report. # # If you would like to disable reporting, uncomment the following lines: #analytics: # reporting_enabled: false

# https://github.com/grafana/loki/blob/main/production/helm/loki/values.yaml # Grafana loki parameters: https://grafana.com/docs/loki/latest/configure/ loki: storage_config: # using tsdb instead of boltdb tsdb_shipper: active_index_directory: /var/loki/tsdb-shipper-active cache_location: /var/loki/tsdb-shipper-cache cache_ttl: 72h # Can be increased for faster performance over longer query periods, uses more disk space shared_store: s3 schemaConfig: configs: - from: 2020-10-24 store: tsdb object_store: s3 schema: v12 index: prefix: index_ period: 24h commonConfig: path_prefix: /var/loki replication_factor: 3 ring: kvstore: store: memberlist storage: bucketNames: chunks: aws-s3-bucket-name ruler: aws-s3-bucket-name type: s3 s3: # endpoint is required if we are using aws IAM user secret access id and key to connect to s3 # endpoint: "s3.amazonaws.com" # Region of the bucket region: s3-bucket-region

Grafana Loki is basically like Prometheus, but for logs. It is a horizontally-scalable, highly-available, multi-tenant log aggregation system inspired by Prometheus. It is designed to store and query logs from multiple sources, making it ideal for use in microservices architectures.

We intend to support a wide variety of destinations. Syslog and Loki are currently under development and are expected to be released in the coming weeks. If you would like your favorite tools to be supported as a destination, vote on this Github discussion!

Loki is a log collation tool from the Grafana family of observability solutions. It aggregates, groups, and labels logs from your applications, helping you troubleshoot problems and monitor activity. Although Loki is a general-purpose tool, it's well-suited to Kubernetes and comes with several Kubernetes-specific features. It automatically scrapes and indexes metadata from your Kubernetes workload objects, such as Pod labels, to accompany your Pod logs.

It's like Prometheus, but for logs. Okay it's not really to do with the Norse or Greek gods, instead Loki is a horizontally-scalable, highly-available, multi-tenant log aggregation system inspired by the open source project Prometheus. Built by Grafana Labs, Loki is designed for ease of use. Instead of indexing the contents of the logs, Loki provides a set of labels for each log stream. The latest update includes query acceleration with Bloom filters, native OTel support, Helm charts, and more. Check out the changelog for all the major changes and deprecations.