We Saved Millions in SSD Costs by Upgrading Our Filesystem

This page summarizes the projects mentioned and recommended in the original post on news.ycombinator.com

Our great sponsors
  • InfluxDB - Power Real-Time Data Analytics at Scale
  • WorkOS - The modern identity platform for B2B SaaS
  • SaaSHub - Software Alternatives and Reviews
  • zfs

    OpenZFS on Linux and FreeBSD

  • After a bit of digging, yes but no.

    So turns out it's a bit more involved than a straight up 80% == bad scenario. ZFS by default divides[1] each vdev (RAIDZ or mirror set) into ~200 allocation regions called metaslabs[2].

    When allocating from a metaslab[3] it will check if the free space in that metaslab is below the threshold. It seems the threshold was changed from 20% free space to 4%[4] at some point.

    However, as noted that threshold is per metaslab. So if the pool is fragmented, even though the overall free space in the pool is above the 4% threshold, there might be metaslabs with less than that free, which will lead to the expensive best-fit search.

    [1]: https://www.delphix.com/blog/delphix-engineering/openzfs-cod...

    [2]: http://dtrace.org/blogs/ahl/2012/11/08/zfs-trivia-metaslabs/

    [3]: https://github.com/openzfs/zfs/blob/master/module/zfs/metasl...

    [4]: https://www.truenas.com/community/threads/zfs-tweak-for-firs...

  • fucking_shell_scripts

    The easiest, most common sense configuration management tool... because you just use fucking shell scripts.

  • Oh there’s this great thing called files.

    You put your config in there and then you can copy them anywhere, even between AWS accounts.

    They work great with fucking shell scripts. https://github.com/brandonhilkert/fucking_shell_scripts

    You can use FSS on docker, kubernetes, bare metal (otherwise known as computers), Windows, Mac, Amiga, etc.

  • InfluxDB

    Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

    InfluxDB logo
  • rlwrap

    A readline wrapper

  • Dropping a link to `rlwrap` in case anyone is not familiar with it:

    https://github.com/hanslub42/rlwrap

    Note that I've never tried it myself with the mysql/mariadb CLI, but I have used it with other tools, and it's brilliant.

NOTE: The number of mentions on this list indicates mentions on common posts plus user suggested alternatives. Hence, a higher number means a more popular project.

Suggest a related project

Related posts