logstash-logback-encoder VS examples

I use the logstash json format.

We're using https://github.com/logfellow/logstash-logback-encoder with logback (on Play Framework, but should work fine on Lambda as well).

For those interested in the details, I've created an example implementation based on Spring-flavoured REST and Logbook+logstash-logback-encoder within my own json-log-filter project for PoC / reference.

what's the difference to https://github.com/logfellow/logstash-logback-encoder ??

Don't use filebeat. Filebeat is for systems that you cannot change logging for. Push logs directly to logstash via logstash appender. Since I'm mainly logback user, there's one directly by logstash at https://github.com/logstash/logstash-logback-encoder. Quick search indicates that there's https://github.com/viskan/logstash-appender/ for log4j also and it seems it also supports MDC abuse as indicated by https://github.com/viskan/logstash-appender/blob/master/src/main/java/com/viskan/log4j/logstash/appender/LogstashAppender.java#L256. By abusing the MDC you won't need to write a processing pattern in logstash to extract metadata from giant blob line as each key in MDC will get assigned additional value, making your records in elastic search more useful.

You can use the Logstash Logback encooder. You mentioned Elk, so there must be a Logstash running somewhere you can connect to with this appender

We need to add traceId and spanId values to the application log. In production we would use the logstash-logback-encoder to generate logs in JSON format and send them to an ELK but for the demo we use this plain text logback layout:

Honeycomb

I haven't used anything else, but I'll gladly shill for https://honeycomb.io.

With all of this in place I went a step further and added Opentelemetry to track the stats of how often the routine was being triggered on Honeycomb.

Events can be used in many meaningful ways. The Event subsystem of B is pretty much a co-evolution of what honeycomb.io offers, but implemented completely differently - it is on bare-metal, and hence a lot cheaper. Because of that, B never subsampled, but always kept a full low of all events anywhere, no exceptions.

It should be noted that this is a very oblique ad for http://honeycomb.io. That in no way impugns the content of the post, and in fact, it's given the content of the post that I feel compelled to point out that, ultimately, this is an ad. Because what is sales and advertising, anyway? It's just a way to get you to buy a product, and you can't do that if you've never even heard about the product. I'm not currently in the market for an observability solution (something something splunk) but if I were, I've now heard of them.

The question though, is why does money ruin everything? The naive questions of an open source zealot to a proprietary software salesperson are one thing, but since we all need money to live, why does money being part of the equation (eg if someone was getting paid to post here) ruin things? Would a "donate to open source" button be more successful if the donate text is "buy me a coffee", "buy me a beer", or "upgrade my beer from Coors light"?

https://xkcd.com/2347/ was and is true, and if we don't figure out a way to change that, I don't really see a future for open source.

Very cool to see honeycomb.io is doing that. I'm about to embark on my distributed tracing learning journey, this makes me want to try honeycomb right away.

Caveat: I work for a vendor in the O11y space (https://honeycomb.io) as a Developer Advocate, however, this advice is generic, not specific to our platform.

I spent Day 2 at the Colony Club to attend OTel Unplugged. This event was sponsored by Lightstep, Honeycomb, New Relic, Splunk, Dynatrace, Crowdstrike, and NGINX. I came into the event not knowing what to expect. I can sometimes clamp up when I’m around folks that I don’t know, but because I was helping with the event check-in, I got to say hello to a number of the attendees, which helped break the ice. And it turns out that there were a lot of names that I recognized from my work in the OTel community, and it was nice to connect in person with folks whom I’d only previously met through Slack or Zoom.

It's no surprise that my apps are mostly written using Sanic as I'm pretty involved with the project. I've been wanting to start testing honeycomb out as well, so it seemed the perfect opportunity to try out.