s5cmd VS aws-cli

s5cmd - Extended character support for s3 compatible backend

I've used a tool in the past called s5cmd to copy millions of objects, and it was strikingly fast: https://github.com/peak/s5cmd

Some folks say https://github.com/peak/s5cmd is faster than the two options above.

RUN mkdir /tmp/output && cd /tmp/output RUN wget --no-check-certificate https://github.com/peak/s5cmd/releases/download/v1.2.1/s5cmd_1.2.1_Linux-64bit.tar.gz RUN tar xvzf s5cmd_1.2.1_Linux-64bit.tar.gz && mv s5cmd /usr/bin/s5cmd && rm -rf /tmp/output && rm s5cmd_1.2.1_Linux-64bit.tar.gz

Not that I’ve seen but you might checkout https://github.com/peak/s5cmd

I like to use https://github.com/peak/s5cmd

Excellent walkthrough, love boto. We’ve recently been using s5cmd which we’ve found is ridiculously faster than boto without any extra boto tricks.

https://github.com/peak/s5cmd

The AWS CLI is a must-have tool if your team relies on Amazon Web Services. It lets you effortlessly interact with AWS services, orchestrate resource management, and automate tasks from the comfort of your terminal. Once you get used to the tool, you'll notice how convenient and quick it is to fit into your processes – especially compared to going through AWS's web-based user interface.

Just for your consideration, the network effect is very real with package managers, too:

https://search.nixos.org/packages?channel=23.05&show=awscli2 is 2.11.27 (even on the "unstable" channel), versus https://formulae.brew.sh/formula/awscli#default that is 2.12.1, which correctly is the most current (https://github.com/aws/aws-cli/tags)

s3fs allows Linux, macOS, and FreeBSD to mount an S3 bucket via FUSE(Filesystem in Userspace). s3fs makes you operate files and directories in S3 bucket like a local file system. s3fs preserves the native object format for files, allowing use of other tools like AWS CLI.

AFAIK it's still a Python package: https://github.com/aws/aws-cli/tree/2.11.6

Check CLI version, it looks like the earliest version that had DEEP_ARCHIVE support was 1.16.133 (https://github.com/aws/aws-cli/commit/55efab102f774ef17bf51ad32e939e4e03a5ff8a)

And this is the init_localstack.sh file content, a unique thing about localstack its that you can move all strings like an aws-cli tool, also the container deletes all the content and config once the container stops, so the script file must create all the resources that you need from Localstack

It can? I created a pull request for this that hasn't been merged yet: https://github.com/aws/aws-cli/pull/6365