Go Filesystem

Open-source Go projects categorized as Filesystem

Top 23 Go Filesystem Projects

  • duf

    Disk Usage/Free Utility - a better 'df' alternative

  • Project mention: Go: What We Got Right, What We Got Wrong | news.ycombinator.com | 2024-01-04

    Not sure these are really popular, but I cannot resist advertising a few utilities written in Go that I regularly use in my daily workflow:

    - gdu: a NCDU clone, much faster on SSD mounts [1]

    - duf: a `df` clone with a nicer interface [2]

    - massren: a `vidir` clone (simpler to use but with fewer options) [3]

    - gotop: a `top` clone [4]

    - micro: a nice TUI editor [5]

    Building this kind of tools in Go makes sense, as the executables are statically compiled and are thus easy to install on remote servers.

    [1]: https://github.com/dundee/gdu

    [2]: https://github.com/muesli/duf

    [3]: https://github.com/laurent22/massren

    [4]: https://github.com/xxxserxxx/gotop

    [5]: https://github.com/zyedidia/micro

  • juicefs

    JuiceFS is a distributed POSIX file system built on top of Redis and S3.

  • Project mention: Data Sync in JuiceFS 1.2: Enhanced Selective Sync and Performance Optimizations | dev.to | 2024-05-17

    In JuiceFS 1.2, we introduced several new features for juicefs sync. We also optimized performance for multiple scenarios to improve users' data synchronization efficiency when dealing with large directories and complex migrations.

  • InfluxDB

    Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

    InfluxDB logo
  • goofys

    a high-performance, POSIX-ish Amazon S3 file system written in Go

  • Project mention: Is Posix Outdated? | news.ycombinator.com | 2023-10-19

    The author needs to ask themselves: in this cloud technology stack, is there POSIX involved somewhere lower down, where I can't access it? The answer is, of course, "yes". The sort of cloud storage systems described all run on top of POSIX APIs. They provide convenience (cost efficiency is more debatable) compared to the POSIX alternative, but that's because they exist at an entirely different conceptual layer (hence the presence of POSIX anyway, just buried).

    Your point about surfacing a POSIX that's actually there but hidden and thus visible to low-level Amazon employees building the S3 service which makes it invisible to S3 end customers is true but isn't the the point of the article. The author is saying there are motivations for a POSIX-like api visible also the end user.

    So your explanation of stack looks like 2 layers: POSIX api <-- AWS S3 built on top of that

    Author's essay is actually talking about 3 layers: POSIX <-- AWS S3 <-- POSIX

    That's why the blog post has the following links to POSIX-on-top-of-S3-objects :

    https://github.com/s3fs-fuse/s3fs-fuse

    https://github.com/kahing/goofys

    https://www.cuno.io/

  • gdu

    Fast disk usage analyzer with console interface written in Go

  • Project mention: Go: What We Got Right, What We Got Wrong | news.ycombinator.com | 2024-01-04

    Not sure these are really popular, but I cannot resist advertising a few utilities written in Go that I regularly use in my daily workflow:

    - gdu: a NCDU clone, much faster on SSD mounts [1]

    - duf: a `df` clone with a nicer interface [2]

    - massren: a `vidir` clone (simpler to use but with fewer options) [3]

    - gotop: a `top` clone [4]

    - micro: a nice TUI editor [5]

    Building this kind of tools in Go makes sense, as the executables are statically compiled and are thus easy to install on remote servers.

    [1]: https://github.com/dundee/gdu

    [2]: https://github.com/muesli/duf

    [3]: https://github.com/laurent22/massren

    [4]: https://github.com/xxxserxxx/gotop

    [5]: https://github.com/zyedidia/micro

  • gocryptfs

    Encrypted overlay filesystem written in Go

  • Project mention: Syncthing: Untrusted Device Encryption | news.ycombinator.com | 2023-12-07

    I'm looking to improve my documents syncing setup. Currently I'm using owncloud, but that seems overkill for just files syncing and it requires maintenance, so I gave Syncthing a look. The "Untrusted device encryption" was not appealing to me because I'm not convinced by the security aspects yet, and also because it is in beta for now. I used gocryptfs [1] in the past and was quite happy with it, so I'm planning to use it on top of Syncthing to have files synced encrypted. As far as I have read this setup (Syncthing + gocryptfs) seems to be used by several people and has already been discussed by gocryptfs' author, who recommended a `-sharedstorage` flag for such use case [2]. Reading [3] I think gocryptfs is more suited for files syncing than cryfs. I'm aware that the metadata (file size, structure, …) of my files are not encrypted but that's a compromise I'm ready to make.

    I would be happy to hear about opinions about this approach.

    [1] https://nuetzlich.net/gocryptfs/

    [2] https://github.com/rfjakob/gocryptfs/issues/549#issuecomment...

    [3] https://www.cryfs.org/comparison

  • mc

    Simple | Fast tool to manage MinIO clusters :cloud:

  • s5cmd

    Parallel S3 and local filesystem execution tool.

  • Project mention: GitHub issues from top Open Source Golang Repositories that you should contribute to | dev.to | 2024-01-15

    s5cmd - Extended character support for s3 compatible backend

  • SaaSHub

    SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives

    SaaSHub logo
  • TMSU

    TMSU lets you tags your files and then access them through a nifty virtual filesystem from any other application.

  • Project mention: Johnny Decimal: A System to Organize Projects | news.ycombinator.com | 2023-09-14

    https://github.com/oniony/TMSU/wiki/FAQ#why-does-tmsu-not-au...

    There are a couple very barebones wrappers around mv and rm, though they could be better (pass through arguments, etc.).

    https://github.com/oniony/TMSU/wiki/Tricks-and-Tips#filesyst...

  • go-fuse

    FUSE bindings for Go

  • goutil

    💪 Helper Utils(700+): int, byte, string, array/slice, map, struct, dump, convert/format, error, web/http, cli/flag, OS/ENV, filesystem, system, test/assert, time and more. Go 常用的一些工具函数:数字,字符串,数组,Map,结构体,反射,文本,文件,错误,时间日期,特殊处理,格式化,常用信息获取等等

  • Project mention: Release v0.6.10 · gookit/goutil - 💪 Helper Utils(700+): int, byte, string, array/slice, map, struct, dump, convert/format, error, web/http, cli/flag, OS/ENV, filesystem, system, test/assert, time and more. | /r/golang | 2023-07-04
  • onedriver

    A native Linux filesystem for Microsoft OneDrive

  • Project mention: What's the most pratical solution for someone who wants to use Linux as their primary OS but often has to deal with Windows-exclusive software and services? | /r/linuxquestions | 2023-12-06

    onedriver is a native Linux filesystem for Microsoft OneDrive: - https://github.com/jstaf/onedriver

  • hubfs

    File system for GitHub & GitLab

  • stargz-snapshotter

    Fast container image distribution plugin with lazy pulling

  • Project mention: Tree-shaking, the horticulturally misguided algorithm | news.ycombinator.com | 2024-04-13

    A lazy chunked delivery strategy like used in the k8s stargz-snapshotter[0] project could be effective here, where it only pulls chunks as needed, but it would probably require wasm platform changes.

    [0] https://github.com/containerd/stargz-snapshotter

  • distribyted

    Torrent client with HTTP, fuse, and WebDAV interfaces. Start exploring your torrent files right away, even zip, rar, or 7zip archive contents!

  • fscrypt

    Go tool for managing Linux filesystem encryption

  • superfile

    Pretty fancy and modern terminal file manager

  • Project mention: FLaNK-AIM Weekly 13 May 2024 | dev.to | 2024-05-13
  • afs

    Abstract File Storage

  • hackpadfs

    An extensible file system abstraction for Go. File systems, composable interfaces, and test suites.

  • enigma

    🔢🔒Enigma: a simple cross-platform encrypted filesystem (by aegistudio)

  • switchboard

    Auto-magic file organisation and routing for all your machines. :open_file_folder: (by Cian911)

  • Project mention: Llama-fs: A self-organizing file system with llama 3 | news.ycombinator.com | 2024-05-26

    I created something very similar, and much simpler, to this in Go not too long ago (without Llama) called Switchboard.

    it's not "self organising" in this sense, but it's an easy way (imo) to organise files across your desktop.

    https://github.com/Cian911/switchboard/

  • juicefs-csi-driver

    JuiceFS CSI Driver

  • Project mention: Ask HN: What distributed file system would you use in 2024? | news.ycombinator.com | 2024-05-10
  • got

    Got is like git, but with an 'o' (by gotvc)

  • Project mention: FLaNK Stack Weekly 2 October 2023 | dev.to | 2023-10-02
  • SaaSHub

    SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives

    SaaSHub logo
NOTE: The open source projects on this list are ordered by number of github stars. The number of mentions indicates repo mentiontions in the last 12 Months or since we started tracking (Dec 2020).

Go Filesystem related posts

  • Data Sync in JuiceFS 1.2: Enhanced Selective Sync and Performance Optimizations

    1 project | dev.to | 17 May 2024
  • Ask HN: What distributed file system would you use in 2024?

    8 projects | news.ycombinator.com | 10 May 2024
  • Smooth Upgrade: Implementation and Usage

    1 project | dev.to | 10 May 2024
  • Simple Tape File System (STFS), a file system for tapes and tar files

    1 project | news.ycombinator.com | 29 Mar 2024
  • South Korea's No.1 Search Engine Chose JuiceFS over Alluxio for AI Storage

    2 projects | dev.to | 18 Jan 2024
  • Go: What We Got Right, What We Got Wrong

    22 projects | news.ycombinator.com | 4 Jan 2024
  • File Integrity and checksums

    1 project | /r/DataHoarder | 9 Dec 2023
  • A note from our sponsor - SaaSHub
    www.saashub.com | 29 May 2024
    SaaSHub helps you find the best software and product alternatives Learn more →

Index

What are some of the best open-source Filesystem projects in Go? This list will help you:

Project Stars
1 duf 12,362
2 juicefs 9,905
3 goofys 5,057
4 gdu 3,348
5 gocryptfs 3,330
6 mc 2,725
7 s5cmd 2,366
8 TMSU 1,992
9 go-fuse 1,955
10 goutil 1,849
11 onedriver 1,808
12 hubfs 1,607
13 stargz-snapshotter 1,060
14 distribyted 1,032
15 fscrypt 865
16 superfile 3,767
17 afs 291
18 hackpadfs 240
19 enigma 223
20 switchboard 197
21 juicefs-csi-driver 195
22 got 128
23 nav 126

Sponsored
SaaSHub - Software Alternatives and Reviews
SaaSHub helps you find the best software and product alternatives
www.saashub.com