JsonPath VS xidel

For this example, we will use JSON files to encode the application's behavior and query it using JSONPath.

As shown above in order to access the root element of a JSON payload we can create variable (data) and assign the $ (using Json Path convention) in the expression tab.

You certainly could implement a recursive traversal, as /u/raevnos suggested, but it might be easier to use something like JsonPath to do it for you.

If you work with REST interfaces for any amount of time, you'll run into "json path". You can extract and manipulate the json received with an API call.

Look into JSONPath and its one Java implementation.

is anyone else using the https://github.com/json-path/JsonPath over the jq route?

I hope we standardize on some jq query language, like we have with a base set of SQL syntax

I use JsonPath for this. This fits my use case of picking apart JSON responses from APIs in Cucumber tests, which is based on textual descriptions and I'm not using Java classes for the mapping.

flatMap — A Map function (can be of Rich type if need be) that extracts data from the input event which is a stringified JSON object. Of the three flapMaps, two of them extract & parse the relevant data into a defined model (the data modelling is based on POJO — plain old java object classes.) using JsonPath, the third flapMap is a RichCoFlatMapFunction that determines after correlating events from the two sources whether it should be forwarded to the sink or not.

JSONPath confers pretty much the same concept onto JSON too with much the same syntax. There are JPath implementations for most languages now.

> https://github.com/json-path/JsonPath

You could try Xidel[1]. It supports JSON, XML and HTML using XPath/XQuery 3.1

It has some extensions to the standard that are pretty nice (JSONiq, CSS selectors, html “template” matching), but you can limit it to just standard XPath/XQuery if you like.

I recommend getting the nightly v .99 build if you give it a try, the stable .98 version is pretty old and I’ve had no issues with .99

1. https://www.videlibri.de/xidel.html

Windows binary of Xidel (https://github.com/benibela/xidel) a commandline tool to download and extract data from HTML pages

Xidel for extraction and pagination

Download an entire subreddit to JSON Lines with Xidel:

I believe that you can't read from a url with batch alone. you'll need a helper app like xidel https://www.videlibri.de/xidel.html

I have not played with this, but it looks like xidel might allow you to do this in Bash. jc also has a URL string parser that could be used in such a script.

I built Xidel for such authenticated requests: