juicefs
gcsfuse
Our great sponsors
juicefs | gcsfuse | |
---|---|---|
42 | 31 | |
9,675 | 1,958 | |
4.1% | 1.4% | |
9.7 | 9.7 | |
3 days ago | 6 days ago | |
Go | Go | |
Apache License 2.0 | Apache License 2.0 |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
juicefs
-
South Korea's No.1 Search Engine Chose JuiceFS over Alluxio for AI Storage
Support for Kerberos keytab files
-
5 Open Source tools written in Golang that you should know about
JuiceFS under the Apache License 2.0, is a high-performance POSIX file system optimized for cloud-native environments. It stores data in Object Storage (e.g., Amazon S3) and metadata in databases like Redis, MySQL, or TiKV. JuiceFS integrates massive cloud storage with big data, machine learning, and AI applications efficiently, akin to local storage. It features full POSIX and Hadoop compatibility, S3 interface, Kubernetes support, and shared file storage for numerous clients. Some cool features are - strong consistency, scalable performance, data encryption, global file locks, and compression with LZ4 or Zstandard.
-
Gcsfuse: A user-space file system for interacting with Google Cloud Storage
If you really expect a file system experience over GCS, please try JuiceFS [1], which scales to 10 billions of files pretty well with TiKV or FoundationDB as meta engine.
PS, I'm founder of JuiceFS.
The architecture image shows GCS and others, so I suspect it does.
-
Google Cloud Storage FUSE
See also: JuiceFS: https://juicefs.com/
Adds a DBMS or key-value store for metadata, making the filesystem much faster (POSIX, small overwrites don't have to replace a full object in the GCS/S3 backend).
Almost certainly a better solution if you want to turn your object storage into a mountable filesystem, with the (big) caveat that you can't access the files directly in the bucket (they are not stored transparently).
- Using S3 as shared storage
-
s3fs-fuse VS juicefs - a user suggested alternative
2 projects | 19 Feb 2023
JuiceFS can do the same thing as s3fs-fuse, but better. Because it supports robust data consistency and caching policies to improve performance.
-
Open source cloud file system. Posix, HDFS and S3 compatible
Atomic file/directory renames/moves is the fundamental feature of JuiceFS, which makes it truely a file system rather than a proxy to S3, please check the docs for all the compatibility details [1].
- An open-source distributed object storage service
-
POSIX Compatibility Comparison among four file system on the cloud
From Juicedata/JuiceFS ! (0ᴗ0✿)
gcsfuse
-
Gcsfuse: A user-space file system for interacting with Google Cloud Storage
It uses FUSE and there's three types of Kernel cache you could use with FUSE (although, it seems like gcsfuse is exposing only one):
1. Cache of file attributes in the Kernel (this is controlled by "stat-cache-ttl" value - https://github.com/GoogleCloudPlatform/gcsfuse/blob/7dc5c7ff...)
-
Google Cloud Storage FUSE
Is this the same gcsfuse that's been around for years, only now with official Google support?
-
RANT: MICROSOFT'S INABILITY TO SUPPORT THEIR OWN HARDWARE IS GOING TO KILL ME
I'm pretty sure the storage for a stopped VM vs a disk image will be the same. Cheaper if you can store your data in GCS bucket? Take a look at GCS-Fuse to mount storage buckets into a VM.
-
FTP Server Backed By Cloud Storage (S3)
Since I started a while back with Google Cloud Storage, I currently use gcsfuse (https://github.com/GoogleCloudPlatform/gcsfuse).
-
Managing GCS buckets with FTP client?
You can use gcsfuse.
-
csi-gcs - CSI driver for mounting Google Cloud Storage buckets
Uses https://github.com/GoogleCloudPlatform/gcsfuse -- last I looked (a long time ago), it was unusably slow with 5000 files.
-
File Systems implemented in Go
gcsfuse - A user-space file system for interacting with Google Cloud Storage.
- You know what I hate? We have the ability to run linux on our Chromebooks but STILL no official support for Google Drive...
What are some alternatives?
google-drive-ocamlfuse - FUSE filesystem over Google Drive
cubefs - cloud-native file store
goofys - a high-performance, POSIX-ish Amazon S3 file system written in Go
s3-benchmark - Measure Amazon S3's performance from any location.
Golang-PDF-to-Image-Converter - This project will help you to convert PDF file to IMAGE using golang.
hdfs - A native go client for HDFS
containers-roadmap - This is the public roadmap for AWS container services (ECS, ECR, Fargate, and EKS).
redisraft - A Redis Module that make it possible to create a consistent Raft cluster from multiple Redis instances.
go-sanitize - :bathtub: Golang library of simple to use sanitation functions
JSON-to-Go - Translates JSON into a Go type in your browser instantly (original)
go-lock - go-lock is a lock library implementing read-write mutex and read-write trylock without starvation