jfq VS Hacker News API

One of the best tools I've found for manipulating JSON via JsonPath syntax is https://jsonata.org

In addition to simple queries that allow you to select one (or multiple) matching nodes, it also provides some helper functions, such as arithmetic, comparisons, sorting, grouping, datetime manipulation, and aggregation (e.g. sum, max, min).

I just discovered JSONata.

I agree that once a project gets a bit more complex, these points are very important.

In our low code platform (https://github.com/dashjoin/platform), you end up writing several JSONata (https://jsonata.org/) snippets for ETL, actions, and visualizations.

These can be tested using junit. All changes to the app can be managed and deployed via GitHub. You can check out our sample app:

GitHub: https://github.com/dashjoin/dashjoin-demo

Learn more at https://jsonata.org/

if anyone's here looking for javascript-language json query-er, I've been using jsonata. found it deep and well-thought out, and not too hard to get my head around the syntax

https://jsonata.org

Another one that comes to mind I've used in grafana https://jsonata.org/

The BigQuery dataset is entirely 3rd party, based on the public HN data (presumably via the HN Firebase API - https://github.com/HackerNews/API), so anyone is free to do this.

Or HN's actual API, which isn't rate limited: https://github.com/HackerNews/API

fwiw, the API (described here https://github.com/HackerNews/API) will just give you json, eg https://hacker-news.firebaseio.com/v0/item/40564831.json?pri...

Hey! This leverages the official HN API (https://github.com/HackerNews/API), no scraping involved. I don't think it's my place to opine on "who owns the posts".

I built a plugin that allows Coze bots to make requests to the Hacker News API.

I recognise the huge amounts of effort involved in this and I applaud the moderators for keeping HN an interesting place to be.

That said, I think it's reasonable for us to have visibility on their manual interventions, and this could be easily surfaced via the Hacker News API (https://github.com/HackerNews/API), if the Story JSON included the values of "contro," "bury," and "gag" fields, which are currently opaque to users of the API

See https://medium.com/hacking-and-gonzo/how-hacker-news-ranking... for more discussion on terminology

Google probably stopped updating BigQuery when they started hosting live Hacker News data in Firebase: https://github.com/HackerNews/API

The live nature of the Firebase data is awesome, but the lack of ability to query is a loss.