structured-text-tools VS csvq

This works well in scripts and logs of all the commands you need to do to reproduce the current state of the system from a scratch install. Also can be used with diff -u and patch, sed, perl, and awk oneliners and structured text tools. You can also capture most of the commands using sudo logging feature but it won't capture the here documents. But for modest size files you can use newlines in echo commands. Note that commands which use redrection should use something like ~~~~ sudo bash -c "echo 'foo' >>file.txt" ~~~~ instead of "sudo echo foo >>file.txt" or "echo foo | sudo tee -a file.txt

TFA is about how to handle csv files with awk. This might be useful in straightforward cases.

For all others I’d recommend to have a look at

https://github.com/dbohdan/structured-text-tools

which lists tools to handle structure text formats

in general, I'd pick something from https://github.com/dbohdan/structured-text-tools

Also, here is a list of structured text tools. You may find some tools there that are helpful in editing configuration files from the command line. Or you can use "diff -u" to create a patch file (you need to save the patch files along with sudo.log) to recreate. Also, use sfdisk --dump and sfdisk --backup to save partition information in a form that can be used to recreate backups.

sure can do, if you already use that shell [1], but personally I like specific tools for specific jobs such as jq [2], fx, csvq [3] etc, there's value in decoupling shells from utils (modularity, speed, innovation etc).

[1] I don't but tempted to try, like its data-types concept

[2] https://jqlang.github.io/jq/

[3] https://github.com/mithrandie/csvq

csvq

There is a way of running SQL-like queries against CSV files.

Lately I have had to do a lot of flat file analysis and tools along these lines have been a godsend. Will check this out.

My go to lately has been csvq (https://mithrandie.github.io/csvq/). Really nice to be able run complicated selects right over a CSV file with no setup at all.

csvq (https://mithrandie.github.io/csvq/)

I usually do this with awk, my largest target files being half a TB in size for a project last year (and far too large to hold entirely in RAM). There are some other utilities like csvq and csvsql both of which let you write SQL-style queries against CSV files, but I'm not sure how they perform on large files. There's a nice list of CSV manipulation tools too if any of those jog your memory.

Apparently, there were many who thought the same thing; Tools to execute SQL against CSV were trdsql, q, csvq, TextQL. They were highly functional, hoewver, had many options and no input completion. I found it just a little difficult to use.

Please check out this tool https://github.com/mithrandie/csvq

If you are looking to query nondb data with sql statements then you may want to check something like https://github.com/mithrandie/csvq (SQL for csv).