Go Scale

Open-source Go projects categorized as Scale | Edit details

Top 6 Go Scale Projects

  • tidb

    TiDB is an open source distributed HTAP database compatible with the MySQL protocol

    Project mention: Comparing Nginx Performance in Bare Metal and Virtual Environments | news.ycombinator.com | 2021-10-29

    I do agree with you in that regard, however, that's also a dangerous line of thinking.

    There are attempts to provide horizontal scalability for RDBMSes in a transparent way, like TiDB https://pingcap.com/ (which is compatible with the MySQL 5.7 drivers), however, the list of functionality that's sacrificed to achieve easily extensible clusters is a long one: https://docs.pingcap.com/tidb/stable/mysql-compatibility

    There are other technologies, like MongoDB, which sometimes are more successful at a clustered configuration, however most of the traditional RDBMSes work best in a leader-follower type of replication scenario, because even those aforementioned systems oftentimes have data consistency issues that may eventually pop up.

    Essentially, my argument is that the lack of good horizontally scalable databases or other data storage solutions is easily explainable by the fact that the problem itself isn't solvable in any easy way, apart from adopting eventual consistency, which is probably going to create more problems than it will solve in case of any pre-existing code that makes assumptions about what ways it'll be able to access data and operate on it: https://en.wikipedia.org/wiki/Fallacies_of_distributed_compu...

    To that end, i'd perhaps like to suggest an alternative: use a single vertically scalable RDBMS instance when possible, with a hot standby if you have the resources for that. Let the architecture around it be horizontally scalable instead, and let it deal with the complexities of balancing the load and dealing with backpressure - introduce a message queue if you must, maybe even an in-memory one for simplicity's sake, or consider an event based architecture where "what needs to be done" is encapsulated within a data structure that can be passed around and applied whenever possible. In my eyes, such solutions can in many cases be better than losing the many benefits of having a single source of truth.

    Alternatively, consider sharding as a possibility, or, alternatively, do some domain driven design, figure out where to draw some boundaries and split your service into multiple ones that cover the domain with which you need to work with. Then you have one DB for sales, one for account management, one for reports and so on, all separated by something as simple as REST interfaces and with rate limits or any of the other mechanisms.

    If, however, neither of those two groups of approaches don't seem to be suitable for the loads that you're dealing with, then you probably have a team of very smart people and a large amount of resources to figure out what will work best.

    To sum up, if there are no good solutions in the space, perhaps that's because the problems themselves haven't been solved yet. Thus, sooner or later, they'll need to be sidestepped and their impact mitigated in whatever capacity is possible.

  • dgraph

    Native GraphQL Database with graph backend

    Project mention: The Best Collected Details on the GraphQL Specification – Overview & Language | dev.to | 2021-12-29

    GraphQL, as defined, is not a programming language and not capable of arbitrary computation. This is important to note, as many of the platforms and services that provide GraphQL APIs could lead one to think that GraphQL is providing much of the functionality in these platforms, when really it is merely the facade and presentation via API of the capabilities of the underlying systems and platforms (re: Hasura, AppSync, Astra, Atlas, Dgraph, Contentful, GraphCMS, etc).

  • SonarLint

    Deliver Cleaner and Safer Code - Right in Your IDE of Choice!. SonarLint is a free and open source IDE extension that identifies and catches bugs and vulnerabilities as you code, directly in the IDE. Install from your favorite IDE marketplace today.

  • serving

    Kubernetes-based, scale-to-zero, request-driven compute (by knative)

    Project mention: Kubernetes-based, scale-to-zero, request-driven compute | reddit.com/r/golang | 2021-11-28
  • heimdall

    An enhanced HTTP client for Go (by gojek)

    Project mention: Any http client framework? | reddit.com/r/golang | 2021-08-31
  • spicedb

    Inspired by Google's Zanzibar paper, SpiceDB is a database system for managing security-critical application permissions.

    Project mention: SpiceDB an extensible runtime based authorisation system with global DB - stop reinventing the authz | reddit.com/r/golang | 2022-01-09
  • hedgedhttp

    Hedged HTTP client which helps to reduce tail latency at scale.

    Project mention: Hedged HTTP package (Tail at Scale) | reddit.com/r/golang | 2021-12-01
NOTE: The open source projects on this list are ordered by number of github stars. The number of mentions indicates repo mentiontions in the last 12 Months or since we started tracking (Dec 2020). The latest post mention was on 2022-01-09.

Go Scale related posts

Index

What are some of the best open-source Scale projects in Go? This list will help you:

Project Stars
1 tidb 30,258
2 dgraph 17,313
3 serving 4,295
4 heimdall 2,170
5 spicedb 1,771
6 hedgedhttp 89
Find remote jobs at our new job board 99remotejobs.com. There are 30 new remote jobs listed recently.
Are you hiring? Post a new remote job listing for free.
OPS - Build and Run Open Source Unikernels
Quickly and easily build and deploy open source unikernels in tens of seconds. Deploy in any language to any cloud.
github.com/nanovms