kibana VS SLF4J

As you might have guessed, I spend a lot of time thinking about application security - almost every day, in fact. At my day job, I'm constantly pondering how to enhance Kibana's security in a scalable manner without overburdening my already hardworking team. Outside of work, I'm equally dedicated to making Secutils.dev even more valuable to fellow engineers looking for better security tools.

These newsletters are among the best sources to stay up-to-date with the latest happenings in JavaScript, web development, and Node.js. They conveniently categorize content into sections like new releases, articles & tutorials, and code & tools. Since I use JavaScript/TypeScript and Node.js extensively, both in my day job and for Secutils.dev, I have to stay informed about developments in these essential tools. Usually, I quickly scan through the newsletter and focus only on the items that grab my attention — it doesn't consume much time but keeps me well-informed.

This is the second part of my reflection sparked by the recent “2023 State of Open Source Security” report from Snyk. It got me thinking about the price we pay for false positives in software security. In my previous post, “The Cost of False Positives in Software Security, Part 1: Small Applications”, I talked about how true and false positives affect smaller applications like Secutils.dev. Now, I want to take the same idea and apply it to a much larger software that’s a big part of my daily work: Kibana.

False positives in security are something that really bothers me, as I happen to work on security for both large applications like Kibana, with hundreds of contributors, and smaller ones like Secutils.dev, where I'm the sole developer.

The increasing complexity of modern systems led to the rise of AIOps (Artificial Intelligence for IT Operations) and observability practices. AIOps leveraged machine learning algorithms to automate problem detection, analysis, and resolution. Observability focused on gaining insights into system behaviour through metrics, logs, and traces. As a result, tools like Prometheus, Grafana, and ELK stack (Elasticsearch, Logstash, Kibana) gained popularity.

There was a discussion on Elastic's Github quite a while ago: https://github.com/elastic/kibana/issues/88956 but I haven't found any related documentation on Elastic's website.

ahh good catch here, I have raised a FR to get this added to Kibana https://github.com/elastic/kibana/issues/157348

A few years ago, my team moved from Java to Go. Working on Go projects, we encountered a wide variety of logging frameworks with different APIs, configuration, and formatting. We soon found ourselves longing for a logging abstraction layer like Java’s slf4j, which had proven invaluable for use in reusable libraries or configuring and debugging production systems. So, not long after moving to Go, we began working toward replacing what we had lost in slf4j.

I'm not familiar with Serilog as I code mostly in Java, use slf4j (logs to stdout) and our apps send logs to Cloudwatch using the task definition's awslogs configuration. I prefer it this way because I can customize the log configurations in my task definitions. Also the default stream name has this format prefix-name/container-name/ecs-task-id so I can easily identify the logs of the task I want to look at. I haven't experienced any downsides with this approach and our apps publish a shit ton of logs. Cloudwatch approach looks like you can customize the stream name?

slf4j is open source. You can look at the code.

Java -> Logback, Log4j2, JDK (Java Util Logging), Slf4j, e.t.c.

slf4j para padronização dos logs;

SLF4J

> My God. Logging in protobuf?

Yes, or any other data format and/or transport protocol.

I'm surprised this is up for debate.

> Logging is the lowest of all debugging utilities - its the first thing you ever do writing software - “hello world”. And, while I admire structural logging, the truth is printing strings remains (truly) the lowest common denominator across software developers.

This sort of comment is terribly miopic. You can have a logging API, and then configure your logging to transport the events anywhere, any way. This is a terribly basic feature and requirement, and one that comes out of the box with some systems. Check how SLF4J[1] is pervasive in Java, and how any SLF4J implementation offers logging to stdout or a local file as a very specific and basic usecase.

It turns out that nowadays most developers write software that runs on many computers that aren't stashed over or under their desks, and thus they need efficient and convenient ways to check what's happening either in a node or in all deployments.

[1] https://www.slf4j.org/