gcsfuse
s3fs
| gcsfuse | s3fs | |
|---|---|---|
| 31 | 7 | |
| 1,981 | 814 | |
| 0.9% | 1.4% | |
| 9.7 | 7.8 | |
| about 20 hours ago | 22 days ago | |
| Go | Python | |
| Apache License 2.0 | BSD 3-clause "New" or "Revised" License |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
gcsfuse
-
Gcsfuse: A user-space file system for interacting with Google Cloud Storage
It uses FUSE and there's three types of Kernel cache you could use with FUSE (although, it seems like gcsfuse is exposing only one):
1. Cache of file attributes in the Kernel (this is controlled by "stat-cache-ttl" value - https://github.com/GoogleCloudPlatform/gcsfuse/blob/7dc5c7ff...)
-
Does Cloud file store have life cycle management feature ?
You're looking at Filestore because your software can only write to a mounted file system? If so, you could mount a Google Cloud Storage bucket with Fuse. I haven't used Fuse myself in production, but it may be worth trying out for your workload.
-
Google Cloud Storage FUSE
Is this the same gcsfuse that's been around for years, only now with official Google support?
https://github.com/GoogleCloudPlatform/gcsfuse
- Suggestions on data transfer between VM instances
-
RANT: MICROSOFT'S INABILITY TO SUPPORT THEIR OWN HARDWARE IS GOING TO KILL ME
I'm pretty sure the storage for a stopped VM vs a disk image will be the same. Cheaper if you can store your data in GCS bucket? Take a look at GCS-Fuse to mount storage buckets into a VM.
-
Ongoing Incident in Google Cloud
Currently being tracked here: https://github.com/GoogleCloudPlatform/gcsfuse/issues/961
s3fs
- Read files from s3 using Pandas/s3fs or AWS Data Wrangler?
- Gcsfuse: A user-space file system for interacting with Google Cloud Storage
-
what's the best python client for AWS automation these days?
- https://github.com/fsspec/s3fs (used by `pandas`, wraps aiobotocore)
- High-level, file-system like interface for S3 with AsyncIO support to replace/extend `boto3`
- Show HN: Query SQLite files stored in S3
-
Getting 403 return code from head_object even with s3:ListBucket permission
I'm using Python's s3fs library to check if a particular file exists in s3 with s3fs.S3FileSystem().exists(path), but I'm getting a Forbidden exception. From the stack trace, I can see it fails when calling s3's head_object method. The documentation for head_object method says:
What are some alternatives?
google-drive-ocamlfuse - FUSE filesystem over Google Drive
goofys - a high-performance, POSIX-ish Amazon S3 file system written in Go
rclone - "rsync for cloud storage" - Google Drive, S3, Dropbox, Backblaze B2, One Drive, Swift, Hubic, Wasabi, Google Cloud Storage, Azure Blob, Azure Files, Yandex Files
juicefs - JuiceFS is a distributed POSIX file system built on top of Redis and S3.
s3www - Serve static files from any S3 compatible object storage services (Let's Encrypt ready)
afero - A FileSystem Abstraction System for Go
aws-sdk-go-v2 - AWS SDK for the Go programming language.
fsnotify - Cross-platform file system notifications for Go.
django-s3file - A lightweight file upload input for Django and Amazon S3
go-systemd - Go bindings to systemd socket activation, journal, D-Bus, and unit files
s3-proxy - S3 Reverse Proxy with GET, PUT and DELETE methods and authentication (OpenID Connect and Basic Auth)