mapstructure VS zap

Maybe https://github.com/mitchellh/mapstructure can do what you want? It has some options for Remainder Values and Omit Empty

I've tried using mapstructure to then marshal the map fields into a struct which mostly works (it struggles with times and custom time types which requires a workaround for each case), but this doesn't feel very idiomatic and requires two passes at marshaling.

Although some orders may have more records maybe a superset can be indentified that you can actually create a struct of it and after gathereing first all values into a map then convert it to a struct maybe using a library like https://github.com/mitchellh/mapstructure . this way you can at least isolate the non structured data only on the data extraction part and the rest of your application can work with well formed structs.

Alternatively, you could try https://github.com/mitchellh/mapstructure if you don't know what your incoming structure is

I'm doing it by validating a map[string]any first then putting it into a structure using mapstructure. It covers most use-cases and offers the most flexibility, at the expense of a bit of performance.

That said, it is possible to do this with JSON using something like https://github.com/tidwall/gjson or if you are fine with the switch statement but don't want to marshal and unmarshal again: https://github.com/mitchellh/mapstructure

https://github.com/mitchellh/mapstructure because we have JSON documents which contain rugged arrays ;-)

I've been using mapstructure.Decode to great effect, but currently can't figure out why a given mapping doesn't work. I'd appreciate it if someone could point out wtf I'm doing wrong or at least in the right direction:

What do you mean by "decode"? I've used https://github.com/mitchellh/mapstructure but that doesn't quite look like what you're doing.

The project currently uses slog package from standard library for logging. But switching to a more advanced logger like zap could offer more flexibility and features.

It's nice to have this in the standard library, but it doesn't solve any existing pain points around structured log metadata and contexts. We use zap [0] and store a zap logger on the request context which allows different parts of the request pipeline to log with things like tenantid, traceId, and correlationId automatically appended. But getting a logger off the context is annoying, leads to inconsistent logging practices, and creates a logger dependency throughout most of our Go code.

[0] https://github.com/uber-go/zap

Kubebuilder, like much of the k8s ecosystem, utilizes zap for logging. Out of the box, the Kubebuilder zap configuration outputs a timestamp for each log, which gets formatted using scientific notation. This makes it difficult for me to read the time of an event just by glancing at it. Personally, I prefer ISO 8601, so let's change it!

What else would you expect from a structured logging package?

To me it absolutely makes sense as the default and standard for 99% of applications, and the API isn't much unlike something like Zap[0] (a popular Go structured logger).

The attributes aren't an "arbitrary" concept, they're a completely normal concept for structured loggers. Groups are maybe less standard, but reasonable nevertheless.

I'm not sure if you're aware that this is specifically a structured logging package. There already is a "simple" logging package[1] in the sodlib, and has been for ages, and isn't particularly fast either to my knowledge. If you want really fast you take a library (which would also make sure to optimize allocations heavily).

[0]: https://pkg.go.dev/go.uber.org/zap

[1]: https://pkg.go.dev/log

And finally for structured logging: https://github.com/uber-go/zap

For logging: I recommend using Uber Zap https://github.com/uber-go/zap It will log stack backtraces and makes it super easy to debug errors when deployed. I typically log in the business logic and not below. And log at the entry for failures to start the system. Maybe not necessary for this example, but it’s an essential piece of any API backend.