s4cmd
mountpoint-s3
| s4cmd | mountpoint-s3 | |
|---|---|---|
| 4 | 17 | |
| 1,350 | 4,017 | |
| 0.3% | 1.8% | |
| 0.0 | 9.5 | |
| 10 months ago | 7 days ago | |
| Python | Rust | |
| Apache License 2.0 | Apache License 2.0 |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
s4cmd
-
Mountpoint – file client for S3 written in Rust, from AWS
https://github.com/bloomreach/s4cmd
- Why am I having such trouble with S3 all of a sudden?
-
Downloading files from S3 with multithreading and Boto3
No mention of how it compares to s4cmd (which like s3cmd is python).
https://github.com/bloomreach/s4cmd
-
AWS EC2 spot instance - write directly to S3
I've used s4cmd in the past. It worked well, and you can specify the level of concurrency.
mountpoint-s3
-
Row Zero and Viewport Data Streaming
... or does "S3 file system" mean https://github.com/awslabs/mountpoint-s3 - a Rust project by AWS Labs that provides "a simple, high-throughput file client for mounting an Amazon S3 bucket as a local file system" ?
- s3m: A CLI for streams of data in S3 buckets
-
S3 Express Is All You Need
Looks like support for S3 Express was merged in with version 1.30 just a few hours ago https://github.com/awslabs/mountpoint-s3/pull/642
-
Gcsfuse: A user-space file system for interacting with Google Cloud Storage
mountpoint-s3 is AWS’ first party solution for mounting s3 buckets as file systems: https://github.com/awslabs/mountpoint-s3
Haven’t used it but it looks cool, if a bit immature.
- Mountpoint for S3
- When would something like this come to ADLS Gen 2?
-
Running Amazon S3 Mountpoint Inside a Container
FROM rust:1.68.0 as Build RUN apt-get update && apt-get install -y \ clang\ cmake \ curl \ fuse \ git \ libfuse-dev \ pkg-config \ && apt-get clean \ && rm -rf /var/lib/apt/lists/* \ && git clone --recurse-submodules https://github.com/awslabs/mountpoint-s3.git \ && cd mountpoint-s3 \ && cargo build --release FROM debian:bullseye-slim RUN apt-get update && apt-get install -y \ ca-certificates \ libfuse-dev \ sudo \ && apt-get clean \ && rm -rf /var/lib/apt/lists/* COPY --from=build /mountpoint-s3/target/release/mount-s3 /usr/local/bin/mount-s3 RUN chmod 777 /usr/local/bin/mount-s3 RUN useradd -ms /bin/bash mount-s3-user \ && echo '%sudo ALL=(ALL) NOPASSWD:ALL' >> /etc/sudoers \ && adduser mount-s3-user sudo USER mount-s3-user
- GitHub - awslabs/mountpoint-s3: A simple, high-throughput file client for mounting an Amazon S3 bucket as a local file system.
-
The inside story on Mountpoint for Amazon S3, a high-performance open source file client
This might be useful with a MinIO server, although not directly supported
What are some alternatives?
s3cmd - Official s3cmd repo -- Command line tool for managing S3 compatible storage services (including Amazon S3 and CloudFront).
s3fs-fuse - FUSE-based file system backed by Amazon S3
s5cmd - Parallel S3 and local filesystem execution tool.
PosixSyncFS - PosixSyncFS is a set of Bash scripts that allow users to create a real POSIX filesystem and sync it to a remote storage bucket for backup and recovery purposes.
youtube-dl - Command-line program to download videos from YouTube.com and other video sites
goofys - a high-performance, POSIX-ish Amazon S3 file system written in Go
you-get - :arrow_double_down: Dumb downloader that scrapes the web
aws-eks-iam-auth-controller - Kubernetes operator which consolidates custom resources into `aws-auth` ConfigMap.
FlexGet - The official FlexGet repository
usbd - User-Space Block Device (USBD) Framework (written in Go)
commute-tube - Copy online media to your USB pen by night and watch it on your daily commute
rclone - "rsync for cloud storage" - Google Drive, S3, Dropbox, Backblaze B2, One Drive, Swift, Hubic, Wasabi, Google Cloud Storage, Azure Blob, Azure Files, Yandex Files