jsoniter VS mapstructure

Since most of the time would be spent decoding json, you could try to cut this time using https://github.com/bytedance/sonic or https://github.com/json-iterator/go, both are drop-in replacements for the stdlib, sonic is faster.

We all know the builtin golang JSON parser is slow.

How about doing comparisons against other implementations?

Like this one: https://github.com/json-iterator/go

Json-iterator (https://github.com/json-iterator/go), you can replace all of encoding/json with this. It does the same thing but it's faster.

Do you want to look at every field all at the same time? If not, you can pick out individual fields. There's other packages such as https://github.com/tidwall/gjson or https://github.com/json-iterator/go that let you pass in paths such as "a.b.c" to extract single fields.

For each Go type used within the config, we generate a separate unmarshaller function. The unmarshallers use json-iterator to process the output from CUE, while tracking the path within the config to the unmarshalled value. This path tracking will allow the function to check if live overrides have been provided on that path and return the override instead.

https://github.com/json-iterator/go an alternative JSON encoding package which allows to stream (flush out) encoded data as soon as it's able to (which is in contrast with the stock package which buffers everything until the encoding is known to be complete and OK).

What to use Easyjson is about the top of the pack and it's straightforward. The downside of efficient tools is that they use code generation to create the code required to turn your structs into json to minimise allocations. This is a manual build step which is annoying. Interestingly json-iterator also uses reflection but it's significantly faster. I suspect black magic.

jsoniter for low level access to JSON encode and decode

Well, one used to have https://github.com/mitchellh/mapstructure which assisted here, but the lib then got abandoned.

Maybe https://github.com/mitchellh/mapstructure can do what you want? It has some options for Remainder Values and Omit Empty

I've tried using mapstructure to then marshal the map fields into a struct which mostly works (it struggles with times and custom time types which requires a workaround for each case), but this doesn't feel very idiomatic and requires two passes at marshaling.

Although some orders may have more records maybe a superset can be indentified that you can actually create a struct of it and after gathereing first all values into a map then convert it to a struct maybe using a library like https://github.com/mitchellh/mapstructure . this way you can at least isolate the non structured data only on the data extraction part and the rest of your application can work with well formed structs.

Alternatively, you could try https://github.com/mitchellh/mapstructure if you don't know what your incoming structure is

I'm doing it by validating a map[string]any first then putting it into a structure using mapstructure. It covers most use-cases and offers the most flexibility, at the expense of a bit of performance.

That said, it is possible to do this with JSON using something like https://github.com/tidwall/gjson or if you are fine with the switch statement but don't want to marshal and unmarshal again: https://github.com/mitchellh/mapstructure

https://github.com/mitchellh/mapstructure because we have JSON documents which contain rugged arrays ;-)

I've been using mapstructure.Decode to great effect, but currently can't figure out why a given mapping doesn't work. I'd appreciate it if someone could point out wtf I'm doing wrong or at least in the right direction: