logstash-logback-encoder VS zipkin

I use the logstash json format.

We're using https://github.com/logfellow/logstash-logback-encoder with logback (on Play Framework, but should work fine on Lambda as well).

For those interested in the details, I've created an example implementation based on Spring-flavoured REST and Logbook+logstash-logback-encoder within my own json-log-filter project for PoC / reference.

what's the difference to https://github.com/logfellow/logstash-logback-encoder ??

Don't use filebeat. Filebeat is for systems that you cannot change logging for. Push logs directly to logstash via logstash appender. Since I'm mainly logback user, there's one directly by logstash at https://github.com/logstash/logstash-logback-encoder. Quick search indicates that there's https://github.com/viskan/logstash-appender/ for log4j also and it seems it also supports MDC abuse as indicated by https://github.com/viskan/logstash-appender/blob/master/src/main/java/com/viskan/log4j/logstash/appender/LogstashAppender.java#L256. By abusing the MDC you won't need to write a processing pattern in logstash to extract metadata from giant blob line as each key in MDC will get assigned additional value, making your records in elastic search more useful.

You can use the Logstash Logback encooder. You mentioned Elk, so there must be a Logstash running somewhere you can connect to with this appender

We need to add traceId and spanId values to the application log. In production we would use the logstash-logback-encoder to generate logs in JSON format and send them to an ELK but for the demo we use this plain text logback layout:

When choosing distributed tracing tools, considerations include your technology stack, business requirements, and monitoring complexity. Zipkin, SkyWalking, and OpenTelemetry are popular distributed tracing solutions, each with its unique features.

From the perspective of the realization of GraphQL infrastructure, the interesting direction is "Finding". How to find the problem? How to find the bottleneck of the system? Distributed Tracing System (DTS) will help answer this question. Distributed tracing is a method of observing requests as they propagate through distributed environments. In our scenario, we have dozens of subgraphs, gateway, and transport layer through which the request goes. We have several tools that can be used to detect the whole lifecycle of the request through the system, e.g. Jaeger, Zipkin or solutions that provided DTS as a part of the solution NewRelic.

Zipkin is a distributed tracing system used for tracking and analyzing how requests move through complex systems, especially in setups with many interconnected services, known as microservices.

Distributed Tracing: Middleware for distributed tracing like Jaeger and Zipkin helps monitor and trace requests as they flow through multiple microservices, aiding in debugging, performance optimization, and understanding the system's behavior.

For microservice tracing, you might want to look at Zipkin [0], or OpenTelemetry [1]

[0] https://zipkin.io/

I’ve not used a self-hosted solution before, but here’s one I found. https://zipkin.io/

> IMO the reason these vendors can and do charge so much is not because telemetry software is hard.

I always saw it as "they are charging for their polished UI/experience"

The UI of https://zipkin.io/ versus DataDog is kind of... not really in the same ballpark?

There are the zipkin https://zipkin.io/ and jaeger https://www.jaegertracing.io/ packages/components you can use both have quickstarts if you consider that to be a beginner's guide.

Zipkin, which was developed by Twitter, is an open source tool for distributed tracing that can also be used to troubleshoot latency issues in your application. While Zipkin is Java-based, py_zipkin is an implementation for Python.