yq VS gron

Install jq and yq, and wrap your commands with | yq -y ..

Confusingly there is another project called yq that does exactly what you're suggesting and it's a preprocessor that converts yaml to json and then used jq. https://github.com/kislyuk/yq

yq

I personally find the yq tool from https://github.com/kislyuk/yq much more useful: it has all the same options and formats as `jq` (as it's really a wrapper around jq). Rather than the `yq` in the OP here where only partial functionality exists.

XML data from the API can be parsed and filtered with xq. There may be multiple ways to get it; first try the yq toolset which includes it.

There is also yq [1], which attempts the same for yaml, toml and xml. (And confusingly also contains a binary named "xq" for querying xml, however with a different syntax)

[1] https://github.com/kislyuk/yq

gron (https://github.com/tomnomnom/gron) to transform it and query and then invert the transformation?

It buffers all of its output statements in memory before writing to stdout:

https://github.com/tomnomnom/gron/blob/master/main.go#L204

Have you tried `gron`?

It converts your nested json into a line by line format which plays better with tools like `grep`

From the project's README:

▶ gron "https://api.github.com/repos/tomnomnom/gron/commits?per_page..." | fgrep "commit.author"

json[0].commit.author = {};

json[0].commit.author.date = "2016-07-02T10:51:21Z";

json[0].commit.author.email = "[email protected]";

json[0].commit.author.name = "Tom Hudson";

https://github.com/tomnomnom/gron

It was suggested to me in HN comments on an article I wrote about `jq`, and I have found myself using it a lot in my day to day workflow

> So all I want is a tool to go from json => line oriented and I will do the rest with the vast library of experience I already have at transformations on the command line.*

The tool for that is likely https://github.com/tomnomnom/gron

If JQ is too much, see GRON &| Miller

gron transforms JSON into discrete assignments to make it easier to grep for what you want https://github.com/tomnomnom/gron

Miller is like awk, sed, cut, join, and sort for data formats such as CSV, TSV, JSON, JSON https://github.com/johnkerl/miller

And jless [1] and gron [2].

This is the first I'm hearing of gron, but adding here for completeness sake. Meanwhile, JSON seems to be becoming a standard for CLI tools. Ideal scenario would be if every CLI tool has a --json flag or something similar, so that jc is not needed anymore.

[1] https://jless.io/

[2] https://github.com/tomnomnom/gron