Our great sponsors
-
I was literally just thinking about this a few days ago. I'm super excited by https://www.nushell.sh/ . I think they are hitting on an order of magnitude improvement paradigm of shells that fit very nicely with the theme of this article.
-
jc
CLI tool and python library that converts the output of popular command-line tools, file-types, and common strings to JSON, YAML, or Dictionaries. This allows piping of output to tools like jq and simplifying automation scripts.
Article from 2019 basically advertises https://github.com/kellyjonbrazil/jc which converts output of many unix cli commands into json, depends on Python
-
Sonar
Write Clean Python Code. Always.. Sonar helps you commit clean code every time. With over 225 unique rules to find Python bugs, code smells & vulnerabilities, Sonar finds the issues while you focus on the work.
-
Related, I am wondering if somebody is aware of a project to wrap cli utilies in a minimal 'TUI', along the lines of this: https://github.com/chriskiehl/Gooey , but staying in the terminal. You'd pass the name of the program to run as the first argument. Haven't thought this through fully.
-
Live editing an awk program is pretty trivial to do in Emacs. I use this occasionally with awk-mode which is ~20 LoC:
-
Sure thing!
https://github.com/kellyjonbrazil/jello
Other languages are superior in their handling of maps/arrays, but Python is just so damned popular now I thought it was a good choice to democratize JSON handling.
https://blog.kellybrazil.com/2020/03/25/jello-the-jq-alterna...
-
murex
A smarter shell and scripting environment with advanced features designed for usability, safety and productivity (eg smarter DevOps tooling)
This is why I wrote murex shell (https://github.com/lmorg/murex), it's an alternative $SHELL, so you'd use it in place of Bash or Zsh, but it's optimised for modern DevOps tools. Which means JSON and YAML are first class citizens.
It's syntax isn't 100% POSIX compatible so there is some new stuff to learn but it works with all the existing POSIX tools and is more readable than AWK and Perl but while also being terse enough to write one liners.
-
> JSON’s design assumes the user can read the entire file into memory
No? The design of most JSON libraries assumes that, but there are perfectly good incremental JSON parsers out there[1–3]. It’s just that people don’t seem to have figured out a good API for not-completely-incremental parsing (please prove me wrong here!), but this applies equally to any structured data format as soon as you want to pull out pieces of data that are nested more than one level down.
The lack of length prefixes in JSON does indeed make a solid parser somewhat more difficult, but you get the ability to author and validate it manually instead. All in all a draw and not because of the incremental parsing thing.
(Tabular or otherwise homogeneous data is indeed reprsented wastefully, but unless the individual records are huge json+gzip is a perfectly serviceable “worse-is-better” solution.)
[1] https://github.com/ICRAR/ijson
-
InfluxDB
Access the most powerful time series database as a service. Ingest, store, & analyze all types of time series data in a fully-managed, purpose-built database. Keep data forever with low-cost storage and superior data compression.
-
> JSON’s design assumes the user can read the entire file into memory
No? The design of most JSON libraries assumes that, but there are perfectly good incremental JSON parsers out there[1–3]. It’s just that people don’t seem to have figured out a good API for not-completely-incremental parsing (please prove me wrong here!), but this applies equally to any structured data format as soon as you want to pull out pieces of data that are nested more than one level down.
The lack of length prefixes in JSON does indeed make a solid parser somewhat more difficult, but you get the ability to author and validate it manually instead. All in all a draw and not because of the incremental parsing thing.
(Tabular or otherwise homogeneous data is indeed reprsented wastefully, but unless the individual records are huge json+gzip is a perfectly serviceable “worse-is-better” solution.)
[1] https://github.com/ICRAR/ijson
-
I wrote something similar to this to query JSON and JSON lines with python instead of awk for text. It’s called Jellex (Jello Explorer) which is a TUI front-end to Jello. Jello is a python analog to JQ.
-
This is how you "integrate" `jc` with Next Generation Shell ( https://github.com/ngs-lang/ngs ):
data = ``jc PROGRAM ARGS ...`` -
Typically informational messages/log message. Ideally it should go do stderr but some applications that intercept the file descriptors consider anything to stderr as error and it is not always possible to change the handling for those applications (e.g. Octopus Deploy[0] does this).
[0]: https://octopus.com/
