jq-zsh-plugin
json-streamer
jq-zsh-plugin | json-streamer | |
---|---|---|
4 | 2 | |
298 | 215 | |
- | - | |
6.0 | 2.4 | |
25 days ago | about 1 year ago | |
Shell | Python | |
MIT License | MIT License |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
jq-zsh-plugin
- Interactive Examples for Learning Jq
-
Analyzing multi-gigabyte JSON files locally
https://github.com/reegnz/jq-zsh-plugin
I find that for big datasets choosing the right format is crucial. Using json-lines format + some shell filtering (eg. head, tail to limit the range, egrep or ripgrep for the more trivial filtering) to reduce the dataset to a couple of megabytes, then use that jq-repl of mine to iterate fast on the final jq expression.
I found that the REPL form factor works really well when you don't exactly know what you're digging for.
json-streamer
- Processing large JSON datasets by streaming
-
Analyzing multi-gigabyte JSON files locally
Might be useful for some - https://github.com/kashifrazzaqui/json-streamer
What are some alternatives?
semi_index - Implementation of the JSON semi-index described in the paper "Semi-Indexing Semi-Structured Data in Tiny Space"
ijson
z-a-readurl - 🌀 An annex delivers the capability to automatically download the newest version of a file to which URL is hosted on a webpage
python-slugify - Returns unicode slugs
json-buffet
awesome-slugify - Python flexible slugify function
lnav - Log file navigator
python-nameparser - A simple Python module for parsing human names into their individual components
reddit_mining
Lark - Lark is a parsing toolkit for Python, built with a focus on ergonomics, performance and modularity.
ClickHouse - ClickHouse® is a free analytics DBMS for big data
pydantic - Data validation using Python type hints