structured-text-tools VS tsv-utils

This works well in scripts and logs of all the commands you need to do to reproduce the current state of the system from a scratch install. Also can be used with diff -u and patch, sed, perl, and awk oneliners and structured text tools. You can also capture most of the commands using sudo logging feature but it won't capture the here documents. But for modest size files you can use newlines in echo commands. Note that commands which use redrection should use something like ~~~~ sudo bash -c "echo 'foo' >>file.txt" ~~~~ instead of "sudo echo foo >>file.txt" or "echo foo | sudo tee -a file.txt

TFA is about how to handle csv files with awk. This might be useful in straightforward cases.

For all others I’d recommend to have a look at

https://github.com/dbohdan/structured-text-tools

which lists tools to handle structure text formats

in general, I'd pick something from https://github.com/dbohdan/structured-text-tools

Also, here is a list of structured text tools. You may find some tools there that are helpful in editing configuration files from the command line. Or you can use "diff -u" to create a patch file (you need to save the patch files along with sudo.log) to recreate. Also, use sfdisk --dump and sfdisk --backup to save partition information in a form that can be used to recreate backups.

If you need just csv/tsv parsing, you can also take a look at https://github.com/eBay/tsv-utils

You might want to look at tsv-utils, or a similar project: https://github.com/eBay/tsv-utils

For the SQL part, but maybe a lot heavier, you can use one of the projects listed on this page: https://github.com/multiprocessio/dsq (No longer maintained, but has links to lots of other projects)

TSV is most often a better format than CSV. Localization, in particular, is a nightmare with CSV.

For anything down and dirty, what's wrong with -F'"'? For anything fancy there are plenty of things like the below.

eBay's TSV Utilities: Command line tools for large, tabular data files. Filtering, statistics, sampling, joins and more.

includes csv to tsv: https://github.com/eBay/tsv-utils

HT: https://simonwillison.net/

As an example, eBay's tsv-utils took full advantage of the GC and performed better than existing programs that had been hand-optimized in C etc.

tsv-utils - Command line csv data manipulation toolkit. D

In the majority of cases you'll be better off with Tab Separated Values over Comma Separated Values. More info here.

May well be overkill for your needs, but I'm a fan of tsv-utils It's fast and enormously flexible, and seems to me a "best of breed" toolset for data mining CSV files (that is what it was written for). https://github.com/eBay/tsv-utils