Open-source projects categorized as Scale | Edit details

Top 19 Scale Open-Source Projects

  • GitHub repo tidb

    TiDB is an open source distributed HTAP database compatible with the MySQL protocol

    Project mention: TiGraph: 8,700x Computing Performance Achieved by Combining Graphs + the RDBMS Syntax | dev.to | 2021-04-05

    The three hackers on the TiGraph team are all top developers in the TiDB community:

  • GitHub repo dgraph

    Native GraphQL Database with graph backend

    Project mention: Need help in choosing a database - Postgres or BadgerDB | reddit.com/r/Database | 2021-05-08

    Dgraph is a highly scalable hyper fast graph database that is distributed, and is built on top of Badger. For consensus, it Raft protocol. (Git repo https://github.com/dgraph-io/dgraph)

  • GitHub repo subsampling-scale-image-view

    Android library (AAR). Highly configurable, easily extendable deep zoom view for displaying huge images without loss of detail. Perfect for photo galleries, maps, building plans etc.

    Project mention: A foss "PDF scanner", alternatives to CamScanner | reddit.com/r/fossdroid | 2020-12-30

    Project: Subsampling Scale Image View Copyright (c) 2015 David Morrissey. License (Apache License Version 2.0) https://github.com/davemorrissey/subsampling-scale-image-view/blob/master/LICENSE

  • GitHub repo citus

    Distributed PostgreSQL as an extension

    Project mention: Some opinionated thoughts on SQL databases | news.ycombinator.com | 2021-03-30

    This depends what you mean by "scaling". Postgres can easily handle 10s of Terabytes on a single server (though this is dependent upon your use case), but when it comes to multiple host sharded databases at the moment MySQL has some more mature options like Vitess [1], where your options in Postgres are things like Citus [2] which just recently open sourced their shard mover, but still have the HA and replication feature behind expensive enterprise pricing.

    For most people not dealing with the multiple terabyte database size, Postgres is what I would definitely recommend. But once you get above that, then there are no easy answers and you would need to look at what YouTube did with Vitess [3]

    [1] https://vitess.io/

    [2] https://www.citusdata.com/

    [3] https://vitess.io/docs/overview/history/

  • GitHub repo fitty

    ✨ Makes text fit perfectly

    Project mention: Modern Card Templates, including fast math typesetting and lists (without add-ons) | reddit.com/r/Anki | 2021-04-13

    There is also the template prettyBasicMaths, which is a basic template that uses KaTeX, which is similar to MathJax. If you're unfamiliar, this is basically what renders the math equations that you input in plain text. The main difference between KaTeX and MathJax is that KaTeX is much, much faster. It also uses a library called fitty, which allows you to make the equations font-size automatically fit your screen. (I wanted to showcase both inlines ($x$) and blocks ($$x$$), which is why I basically put the answer into the question, in practice this is completely nonsensical of course)

  • GitHub repo tonal

    A functional music theory library for Javascript

    Project mention: Do you want to learn music theory? | reddit.com/r/telecaster | 2021-02-11

    For the music theory stuff I'm using TonalJS along with my own extensions

  • GitHub repo heimdall

    An enhanced HTTP client for Go (by gojek)

    Project mention: Can you share some Go package that you think has high quality clean code? | reddit.com/r/golang | 2021-04-13

    Heimdall HTTP Client

  • GitHub repo middleware

    TrueNAS CORE/Enterprise/SCALE Middleware Git Repository

    Project mention: Mercredi Tech - 2021-05-05 | reddit.com/r/france | 2021-05-05
  • GitHub repo Brighter

    Command Dispatcher, Processor, and Distributed Task Queue

    Project mention: Exploring vertical slices in dotnet core | dev.to | 2021-03-05

    There are some other libraries to implement this, even from scratch, but at this time I'm using the same Mediatr library, implementing IRquest and IRequestHandler interfaces.

  • GitHub repo flyte

    Accelerate your ML and Data workflows to production. Flyte is a production grade orchestration system for your Data and ML workloads. It has been battle tested at Lyft, Spotify, freenome and others and truly open-source.

    Project mention: [P] Accelerate Your Machine Learning and Data Workflows to Production using Flyte! | reddit.com/r/MachineLearning | 2021-04-07

    You can check out Flyte here: https://github.com/flyteorg/flyte, and here is the website.

  • GitHub repo SmoothRefreshLayout


  • GitHub repo typesettings

    A Sass or Stylus toolkit that sets type in Ems based on modular scale, vertical rhythm, and responsive ratio based headlines.

    Project mention: CSS Deep | dev.to | 2021-02-26

    ianrose/typesettings - A Sass or Stylus toolkit that sets type in Ems based on modular scale, vertical rhythm, and responsive ratio based headlines.

  • GitHub repo Scale

    :flags: Unit converter in Swift (by onmyway133)

  • GitHub repo Plezi

    Plezi - the Ruby framework for realtime web-apps, websockets and RESTful HTTP

  • GitHub repo polybox

    An itty-bitty PaaS that uses git push to deploy micro-services and websites on your own servers, like Okurrr!!!

    Project mention: Zero-downtime Heroku-style deployments with Git and Docker | news.ycombinator.com | 2021-04-25

    For a simpler option without Docker, but still with GIT Heroku-style, check out Polybox[0].

    Polybox[0] is an itty-bitty PaaS that uses git push to deploy micro-services and websites on your own servers.


  • GitHub repo nitroml

    NitroML is a modular, portable, and scalable model-quality benchmarking framework for Machine Learning and Automated Machine Learning (AutoML) pipelines.

    Project mention: Launch HN: MindsDB (YC W20) – Machine Learning Inside Your Database | news.ycombinator.com | 2021-02-19

    The benchmarking challenges you are facing are pretty common in the AutoML community. My colleagues and I at Google Research are trying to solve this with https://github.com/google/nitroml. It's still super early days (no CI yet), but I think it could help your team benchmark on a set of open standard benchmark tasks as we open source more of the system.

  • GitHub repo frettler

    Java utility for creating scales and chords and displaying them with any fretboard layout

    Project mention: Frettler - Open source fretboard utility now supports lefties | reddit.com/r/guitars | 2021-05-02

    Frettler, my open source, command line fretboard program, now has support for progressions, left handed instruments and preset, non standard, tunings.

  • GitHub repo scaling-nodejs-with-docker

    Sample Node.js project showing how to scale horizontally with Docker

    Project mention: How does docker help with load balancing? | reddit.com/r/node | 2021-04-21

    This article covers horizontal scaling Node.js apps in Docker. There's also an accompanying repository to see how it's done https://github.com/Maximization/scaling-nodejs-with-docker

  • GitHub repo super-comic-slicer

    An artist's long comic page cutter into their desired webcomic platform.

    Project mention: How to make site load faster? (w/ Webpack) | reddit.com/r/react | 2021-03-23

    I heard of "code splitting". I'm not sure if I have implemented correctly. Here is the site: https://supercomicslicer.netlify.app/ Here is my repo: https://github.com/hyfydistro/super-comic-slicer

NOTE: The open source projects on this list are ordered by number of github stars. The number of mentions indicates repo mentiontions in the last 12 Months or since we started tracking (Dec 2020). The latest post mention was on 2021-05-08.


What are some of the best open-source Scale projects? This list will help you:

Project Stars
1 tidb 27,630
2 dgraph 15,960
3 subsampling-scale-image-view 6,756
4 citus 4,805
5 fitty 3,224
6 tonal 1,990
7 heimdall 1,832
8 middleware 1,785
9 Brighter 1,419
10 flyte 1,375
11 SmoothRefreshLayout 1,175
12 typesettings 380
13 Scale 324
14 Plezi 239
15 polybox 81
16 nitroml 32
17 frettler 31
18 scaling-nodejs-with-docker 12
19 super-comic-slicer 1