Managing outdated pull requests is time-consuming. Mergify's Merge Queue automates your pull request management & merging. It's fully integrated to GitHub & coordinated with any CI. Start focusing on code. Try Mergify for free. Learn more →
Top 23 C++ Filesystem Projects
-
Project mention: Gcsfuse: A user-space file system for interacting with Google Cloud Storage | news.ycombinator.com | 2023-09-06
-
I guess you can checkout this library https://github.com/itinance/react-native-fs it will allow you to download your photo and you will need to spend some time to find out how exactly you need to specify the path of this photo to be downloaded to. There are a lot of predefined paths that you might find helpful, like RNFS.DocumentDirectoryPath or RFNS.ExternalStorageDirectoryPath where you might look for your album
-
Mergify
Tired of breaking your main and manually rebasing outdated pull requests?. Managing outdated pull requests is time-consuming. Mergify's Merge Queue automates your pull request management & merging. It's fully integrated to GitHub & coordinated with any CI. Start focusing on code. Try Mergify for free.
-
Project mention: The Next Gen Database Servers Powering Let's Encrypt(2021) | news.ycombinator.com | 2023-09-17
Like most people on r/homelab, it started out with Plex. Rough timeline/services below:
0. Got a Synology DS413 with 4x WD Red 3TB drives. Use Playstation Media Server to stream videos from it. Eventually find some Busybox stuff to add various functionality to the NAS, but it had a habit of undoing them periodically, which was frustrating. I also experienced my first and (knock on wood) only drive failure during this time, which concluded without fanfare once the faulty drive was replaced, and the array repaired itself.
1. While teaching self Python as an Electrical Distribution Engineer at a utility, I befriended the IT head, who gave me an ancient (I think Nehalem? Quad-core Xeon) Dell T310. Promptly got more drives, totaling 7, and tried various OS / NAS platforms. I had OpenMediaVault for a while, but got tired of the UI fighting me when I knew how to do things in shell, so I switched to Debian (which it's based on anyway). Moved to MergerFS [0] + SnapRAID [1] for storage management, and Plex for media. I was also tinkering with various Linux stuff on it constantly.
1.1 Got tired of my tinkering breaking things and requiring troubleshooting/fixing (in retrospect, this provided excellent learning), so I installed Proxmox, reinstalled Debian, and made a golden image with everything set up as desired so I could easily revert.
1.2 A friend told me about Docker. I promptly moved Plex over to it, and probably around this time also got the *Arr Stack [2] going.
2. Got a Supermicro X9DRi-LN4F+ in a 2U chassis w/ 12x 3.5" bays. Got faster/bigger CPUs (E5-2680v2), more RAM, more drives, etc. Shifted container management to Docker Compose. Modded the BIOS to allow it to boot from a NVMe drive on a PCIe adapter.
2.1 Shifted to ZFS on Debian. Other than DKMS occasionally losing its mind during kernel upgrades, this worked well.
2.2 Forked [3] some [4] Packer/Ansible projects to suit my needs, made a VM for everything. NAS, Dev, Webserver, Docker host, etc. Other than outgrowing (IMO) MergerFS/SnapRAID, honestly at this point I could have easily stopped, and could to this day revert back to this setup. It was dead reliable and worked extremely well. IIRC I was also playing with Terraform at this time.
2.3 Successfully broke into tech (Associate SRE) as a mid-career shift, due largely (according to the hiring manager) to what I had done with my homelab. Hooray for hobbies paying off.
3. Got a single Dell R620. I think the idea was to install either pfSense or VyOS on it, but that never came to fruition. Networking was from a Unifi USG (their tiny router + firewall + switch) and 8-port switch, with some AC Pro APs.
4. Got two more R620s. Kubernetes all the things. Each one runs Proxmox in a 3-node cluster with two VMs - a control plane, and worker.
4.0.1 Perhaps worth noting here that I thoroughly tested my migration plan via spinning up some VMs in, IIRC, Digital Ocean that mimicked my home setup. I successfully ran it twice, which was good enough for me.
4.1 Played with Ceph via Rook, but a. disliked (and still to this day) running storage for everything out of K8s b. kept getting clock skew between nodes. Someone on Reddit mentioned it was my low-power C-state settings, but since that was saving me something like ~50 watts/node, I didn't want to deal with the higher power/heat. I landed on Longhorn [5] for cluster storage (i.e. anything that wasn't being handled by the ZFS pool), which was fine, but slow. SATA SSDs (used Intel enterprise drives with PLP, if you're wondering) over GBe aren't super fast, but they should be able to exceed 30 MBps.
4.1.1 Again, worth noting that I spent literally a week poring over every bit of Ceph documentation I could find, from the Red Hat stuff to random Wikis and blog posts. It's not something you just jump into, IMO, and most of the horror stories I read boiled down to "you didn't follow the recommended practices."
5. Got a newer Supermicro, an X11SSH-F, thinking that it would save power consumption over the older dual-socket I had for the NAS. It turned out to not make a big difference. For some reason I don't recall, I had a second X9DRi-LN4F+ mobo, so I sold the other one with the faster CPUs, bought some cheaper CPUs for the other one, and bought more drives for it. It's now a backup target that boots up daily to ingest ZFS snapshots. I have 100% on-site backups for everything. Important things (i.e. anything that I can't get from a torrent) are also off-site.
6. Got some Samsung PM863 NVMe SSDs mounted on PCIe adapters for the Dells, and set up Ceph, but this time handled by Proxmox. It's dead easy, and for whatever reason isn't troubled by the same clock skew issues as I had previously. Still in the process of shifting cluster storage from Longhorn, but I have been successfully using Ceph block storage as fast (1 GBe, anyway - a 10G switch is on the horizon) storage for databases.
So specifically, you asked what I do with the hardware. What I do, as far as my family is concerned, is block ads and serve media. On a more useful level, I try things out related to my job, most recently database-related (I moved from SRE to DBRE a year ago). I have MySQL and Postgres running, and am constantly playing with them. Can you actually do a live buffer pool resize in MySQL? (yes) Is XFS actually faster than ext4 for large DROP TABLE operations? (yes, but not by much) Is it faster to shut down a MySQL server and roll back to a previous ZFS snapshot than to rollback a big transaction? (often yes, although obviously a full shutdown has its own problems) Does Postgres suffer from the same write performance issue as MySQL with random PKs like UUIDv4, despite not clustering by default? (yes, but not to the same extent - still enough to matter, and you should use UUIDv7 if you absolutely need them)
I legitimately love this stuff. I could quite easily make do without a fancy enclosed rack and multiple servers, but I like them, so I have them. The fact that it tends to help my professional growth out at the same time is a bonus.
[0]: https://github.com/trapexit/mergerfs
[3]: https://github.com/stephanGarland/packer-proxmox-templates
[4]: https://github.com/stephanGarland/ansible-initial-server
[5]: https://longhorn.io
-
Project mention: TabFS – a browser extension that mounts the browser tabs as a filesystem | news.ycombinator.com | 2023-02-18
Something in a similar vein: BTFS, mount a torrent file (or magnet link) as a read only drive https://github.com/johang/btfs
It's really good, and use it a lot of times
-
you have a couple options. if you’re using ext4, fscrypt might be your best bet. if not, a FUSE-based encryption option is probably your only option. ecryptfs, gocryptfs and encfs are good, but i personally prefer cryfs. it doesn’t have an archwiki page, but can be installed from the repos and instructions for use can be found on its webpage
-
OR to attempt to recover data from the old drive, you can use a USB and boot into a Linux Live instance, install apfs-fuse to mount the partition if it is not encrypted. Here's a tutorial to boot into Ubuntu Live. And here is a link to the github repository for apfs-fuse.
-
filesystem
An implementation of C++17 std::filesystem for C++11 /C++14/C++17/C++20 on Windows, macOS, Linux and FreeBSD.
-
SonarCloud
Analyze your C and C++ projects with just one click.. SonarCloud, a cloud-based static analysis tool for your CI/CD workflows, offers a one-click automatic analysis of C and C++ projects hosted on GitHub. Zero configuration and free for open-source projects! Analyze free.
-
Project mention: Help! Does anyone know how to install johncena141 games on linux? | /r/LinuxCrackSupport | 2023-07-01
on a fresh install all you need is dwarfs https://github.com/mhx/dwarfs and libopenal1
-
Project mention: Has anyone gotten tensorflow_io to work on their M1 Mac? As far as I can tell there are no solutions. | /r/tensorflow | 2023-07-23
Been stuck on this for days. The issue is mentioned in this year old thread, with multiple solutions that dont end up working. https://github.com/tensorflow/io/issues/1625 . If we're on Mac are we just SOL? Never encountered this type of problem before where you straight up cant work because your Mac is incompatible with the necessary software.
-
There's cppcryptfs & securefs as Cryptomator alternatives too. SiriKali is an option for a GUI that works nicely with them.
-
I’ve tried to make this library simple and correct: https://github.com/e-dant/watcher
-
-
-
Project mention: In Windows, you can use Winrar to extract split rar files and delete them as you go if you don't have enough disk space. Is there a program that lets you do this in Linux without wine? | /r/linux | 2022-10-07
fuse, mount the rar file https://github.com/google/fuse-archive (or https://github.com/hasse69/rar2fs)
-
-
-
yes I got Doom running on my custom OS.
-
there's a project from Google called mount-zip where you can mount zips (and it actually works really well even with 1TB zips)
-
-
Also see: Experimental: A rust library to monitor filesystem 🪛 and more in windows
-
Project mention: GitHub - BiltuDas1/qsort: Quick File Sorter: A quicker way to organize files for linux | /r/linux | 2023-05-13
-
-
-
InfluxDB
Collect and Analyze Billions of Data Points in Real Time. Manage all types of time series data in a single, purpose-built database. Run at any scale in any environment in the cloud, on-premises, or at the edge.
NOTE:
The open source projects on this list are ordered by number of github stars.
The number of mentions indicates repo mentiontions in the last 12 Months or
since we started tracking (Dec 2020).
The latest post mention was on 2023-09-17.
C++ Filesystem related posts
- Has anyone gotten tensorflow_io to work on their M1 Mac? As far as I can tell there are no solutions.
- R2 slow PUT file transfer
- Help! Does anyone know how to install johncena141 games on linux?
- A.Total.War.Saga.THRONES.OF.BRITANNIA-TENOKE
- Podman and S3 Storage Driver (Audiobookshelf)
- How do I Password-Protect a single folder?
- iOS - save photo in an app specific album
-
A note from our sponsor - Mergify
blog.mergify.com | 21 Sep 2023
Index
What are some of the best open-source Filesystem projects in C++? This list will help you:
| Project | Stars | |
|---|---|---|
| 1 | s3fs-fuse | 7,477 |
| 2 | react-native-fs | 4,776 |
| 3 | mergerfs | 3,407 |
| 4 | btfs | 3,111 |
| 5 | cryfs | 1,842 |
| 6 | apfs-fuse | 1,506 |
| 7 | filesystem | 1,161 |
| 8 | dwarfs | 1,158 |
| 9 | io | 664 |
| 10 | securefs | 664 |
| 11 | watcher | 606 |
| 12 | sparsebundlefs | 292 |
| 13 | fatcat | 264 |
| 14 | fuse-archive | 106 |
| 15 | loggedfs | 101 |
| 16 | hypergrep | 96 |
| 17 | MaslOS | 77 |
| 18 | mount-zip | 76 |
| 19 | 11Zip | 75 |
| 20 | fsfilter-rs | 29 |
| 21 | qsort | 15 |
| 22 | tff | 2 |
| 23 | WinXSearch | 1 |
Collect and Analyze Billions of Data Points in Real Time
Manage all types of time series data in a single, purpose-built database. Run at any scale in any environment in the cloud, on-premises, or at the edge.
www.influxdata.com