csvq VS aws-cli

sure can do, if you already use that shell [1], but personally I like specific tools for specific jobs such as jq [2], fx, csvq [3] etc, there's value in decoupling shells from utils (modularity, speed, innovation etc).

[1] I don't but tempted to try, like its data-types concept

[2] https://jqlang.github.io/jq/

[3] https://github.com/mithrandie/csvq

csvq

There is a way of running SQL-like queries against CSV files.

Lately I have had to do a lot of flat file analysis and tools along these lines have been a godsend. Will check this out.

My go to lately has been csvq (https://mithrandie.github.io/csvq/). Really nice to be able run complicated selects right over a CSV file with no setup at all.

csvq (https://mithrandie.github.io/csvq/)

I usually do this with awk, my largest target files being half a TB in size for a project last year (and far too large to hold entirely in RAM). There are some other utilities like csvq and csvsql both of which let you write SQL-style queries against CSV files, but I'm not sure how they perform on large files. There's a nice list of CSV manipulation tools too if any of those jog your memory.

Apparently, there were many who thought the same thing; Tools to execute SQL against CSV were trdsql, q, csvq, TextQL. They were highly functional, hoewver, had many options and no input completion. I found it just a little difficult to use.

Please check out this tool https://github.com/mithrandie/csvq

If you are looking to query nondb data with sql statements then you may want to check something like https://github.com/mithrandie/csvq (SQL for csv).

The AWS CLI is a must-have tool if your team relies on Amazon Web Services. It lets you effortlessly interact with AWS services, orchestrate resource management, and automate tasks from the comfort of your terminal. Once you get used to the tool, you'll notice how convenient and quick it is to fit into your processes – especially compared to going through AWS's web-based user interface.

Just for your consideration, the network effect is very real with package managers, too:

https://search.nixos.org/packages?channel=23.05&show=awscli2 is 2.11.27 (even on the "unstable" channel), versus https://formulae.brew.sh/formula/awscli#default that is 2.12.1, which correctly is the most current (https://github.com/aws/aws-cli/tags)

s3fs allows Linux, macOS, and FreeBSD to mount an S3 bucket via FUSE(Filesystem in Userspace). s3fs makes you operate files and directories in S3 bucket like a local file system. s3fs preserves the native object format for files, allowing use of other tools like AWS CLI.

AFAIK it's still a Python package: https://github.com/aws/aws-cli/tree/2.11.6

Check CLI version, it looks like the earliest version that had DEEP_ARCHIVE support was 1.16.133 (https://github.com/aws/aws-cli/commit/55efab102f774ef17bf51ad32e939e4e03a5ff8a)

And this is the init_localstack.sh file content, a unique thing about localstack its that you can move all strings like an aws-cli tool, also the container deletes all the content and config once the container stops, so the script file must create all the resources that you need from Localstack

It can? I created a pull request for this that hasn't been merged yet: https://github.com/aws/aws-cli/pull/6365