zap VS GJSON

The project currently uses slog package from standard library for logging. But switching to a more advanced logger like zap could offer more flexibility and features.

It's nice to have this in the standard library, but it doesn't solve any existing pain points around structured log metadata and contexts. We use zap [0] and store a zap logger on the request context which allows different parts of the request pipeline to log with things like tenantid, traceId, and correlationId automatically appended. But getting a logger off the context is annoying, leads to inconsistent logging practices, and creates a logger dependency throughout most of our Go code.

[0] https://github.com/uber-go/zap

Kubebuilder, like much of the k8s ecosystem, utilizes zap for logging. Out of the box, the Kubebuilder zap configuration outputs a timestamp for each log, which gets formatted using scientific notation. This makes it difficult for me to read the time of an event just by glancing at it. Personally, I prefer ISO 8601, so let's change it!

What else would you expect from a structured logging package?

To me it absolutely makes sense as the default and standard for 99% of applications, and the API isn't much unlike something like Zap[0] (a popular Go structured logger).

The attributes aren't an "arbitrary" concept, they're a completely normal concept for structured loggers. Groups are maybe less standard, but reasonable nevertheless.

I'm not sure if you're aware that this is specifically a structured logging package. There already is a "simple" logging package[1] in the sodlib, and has been for ages, and isn't particularly fast either to my knowledge. If you want really fast you take a library (which would also make sure to optimize allocations heavily).

[0]: https://pkg.go.dev/go.uber.org/zap

[1]: https://pkg.go.dev/log

And finally for structured logging: https://github.com/uber-go/zap

For logging: I recommend using Uber Zap https://github.com/uber-go/zap It will log stack backtraces and makes it super easy to debug errors when deployed. I typically log in the business logic and not below. And log at the entry for failures to start the system. Maybe not necessary for this example, but it’s an essential piece of any API backend.

Someone made a benchmark of serialization libraries in go [1], and I was surprised to see gobs is one of the slowest ones, specially for decoding. I suspect part of the reason is that the API doesn't not allow reusing decoders [2]. From my explorations it seems like both JSON [3], message-pack [4] and CBOR [5] are better alternatives.

By the way, in Go there are a like a million JSON encoders because a lot of things in the std library are not really coded for maximum performance but more for easy of usage, it seems. Perhaps this is the right balance for certain things (ex: the http library, see [6]).

There are also a bunch of libraries that allow you to modify a JSON file "in place", without having to fully deserialize into structs (ex: GJSON/SJSON [7] [8]). This sounds very convenient and more efficient that fully de/serializing if we just need to change the data a little.

--

1: https://github.com/alecthomas/go_serialization_benchmarks

2: https://github.com/golang/go/issues/29766#issuecomment-45492...

--

3: https://github.com/goccy/go-json

4: https://github.com/vmihailenco/msgpack

5: https://github.com/fxamacker/cbor

--

6: https://github.com/valyala/fasthttp#faq

--

7: https://github.com/tidwall/gjson

8: https://github.com/tidwall/sjson

```

I don't think there is a way to sort an array, though. However, there is an option to have keys sorted. Personally, I don't think there is much annoyance in that. One could just pipe `jj` output to `sort | uniq -c`.

[0]: https://github.com/tidwall/gjson/blob/master/SYNTAX.md

I’m using GJSON, so far so good!

If the fields you need are just a small subset of the whole json, maybe https://github.com/tidwall/gjson might be of use to read only those (using jsonpath) without needing to create complete corresponding structs.

Thank you for the reminder, it's never too much of it :) Didn't say it, but the code was pprof-iled many times and i can really say it's well optimized. I use own libraries with on-the-fly equations (sums, avgs, emas, stds, ...) wherever possible and also made custom json parser as json messages are in fixed format, so the parser is about 10x faster than gjson. I optimized it to the point that I avoided using maps, and rather iterate via slice where ever possible.

You are right, for learning purposes this fit my needs, but I can imagine an approach similar to this repo: https://github.com/tidwall/gjson

https://github.com/tidwall/gjson is a relatively sensible library if this is something you need to deal with and the structure is actually unknowable.