Go Database

Open-source Go projects categorized as Database | Edit details
Related topics: #Go #Golang #SQL #MySQL #Postgresql

Top 23 Go Database Projects

  • GitHub repo prometheus

    The Prometheus monitoring system and time series database.

    Project mention: My media server dashboard, been pretty happy with it for a while, trying to decided if I need any other info on it | reddit.com/r/homelab | 2021-11-29

    Are you looking for general container metrics, or the media server-specific metrics OP has? If you're just looking for general metrics, I just made a blog post tutorial on this, I can PM you if interested but I don't want to spam/self promote here. The three basic components are Grafana, Prometheus, and node exporter

  • GitHub repo etcd

    Distributed reliable key-value store for the most critical data of a distributed system

    Project mention: etcd | reddit.com/r/JavaOnTheEdge | 2021-11-07
  • Nanos

    Run Linux Software Faster and Safer than Linux with Unikernels.

  • GitHub repo tidb

    TiDB is an open source distributed HTAP database compatible with the MySQL protocol

    Project mention: Comparing Nginx Performance in Bare Metal and Virtual Environments | news.ycombinator.com | 2021-10-29

    I do agree with you in that regard, however, that's also a dangerous line of thinking.

    There are attempts to provide horizontal scalability for RDBMSes in a transparent way, like TiDB https://pingcap.com/ (which is compatible with the MySQL 5.7 drivers), however, the list of functionality that's sacrificed to achieve easily extensible clusters is a long one: https://docs.pingcap.com/tidb/stable/mysql-compatibility

    There are other technologies, like MongoDB, which sometimes are more successful at a clustered configuration, however most of the traditional RDBMSes work best in a leader-follower type of replication scenario, because even those aforementioned systems oftentimes have data consistency issues that may eventually pop up.

    Essentially, my argument is that the lack of good horizontally scalable databases or other data storage solutions is easily explainable by the fact that the problem itself isn't solvable in any easy way, apart from adopting eventual consistency, which is probably going to create more problems than it will solve in case of any pre-existing code that makes assumptions about what ways it'll be able to access data and operate on it: https://en.wikipedia.org/wiki/Fallacies_of_distributed_compu...

    To that end, i'd perhaps like to suggest an alternative: use a single vertically scalable RDBMS instance when possible, with a hot standby if you have the resources for that. Let the architecture around it be horizontally scalable instead, and let it deal with the complexities of balancing the load and dealing with backpressure - introduce a message queue if you must, maybe even an in-memory one for simplicity's sake, or consider an event based architecture where "what needs to be done" is encapsulated within a data structure that can be passed around and applied whenever possible. In my eyes, such solutions can in many cases be better than losing the many benefits of having a single source of truth.

    Alternatively, consider sharding as a possibility, or, alternatively, do some domain driven design, figure out where to draw some boundaries and split your service into multiple ones that cover the domain with which you need to work with. Then you have one DB for sales, one for account management, one for reports and so on, all separated by something as simple as REST interfaces and with rate limits or any of the other mechanisms.

    If, however, neither of those two groups of approaches don't seem to be suitable for the loads that you're dealing with, then you probably have a team of very smart people and a large amount of resources to figure out what will work best.

    To sum up, if there are no good solutions in the space, perhaps that's because the problems themselves haven't been solved yet. Thus, sooner or later, they'll need to be sidestepped and their impact mitigated in whatever capacity is possible.

  • GitHub repo cockroach

    CockroachDB - the open source, cloud-native distributed SQL database.

    Project mention: Stacked changes: how FB and Google engineers stay unblocked and ship faster | news.ycombinator.com | 2021-11-17

    I'm surprised Reviewable[0] hasn't come up in this discussion. It does a great job of allowing stacked code reviews and even handles rebases nicely; the reviewer sees the diff between commit #1 and commit #1' (prime = after rebase).

    CockroachDB[1] has been using it since very early in the project.

    [0] https://reviewable.io/

    [1] https://github.com/cockroachdb/cockroach

  • GitHub repo InfluxDB

    Scalable datastore for metrics, events, and real-time analytics

    Project mention: How to stream IoT data into Excel | reddit.com/r/IOT | 2021-11-11

    I would suggest using something better suited to IoT data than ... a spreadsheet. I'd recommend looking at one of the Time Series Databases for this. 1) QuestDB or 2) InfluxDB as these are much better suited to streaming data.

  • GitHub repo dgraph

    Native GraphQL Database with graph backend

    Project mention: DGraph Data Modeling: Part 1 - Timestamps | dev.to | 2021-11-26

    In my opinion, DGraph is the best competition to Firestore there is (and much much more). Imagine Firestore with Subscriptions, plus a Graph Database for complex searching, plus any kind of relational data you can think of, wrapped in a GraphQL interface, or a backend custom language called DQL.

  • GitHub repo jaeger

    CNCF Jaeger, a Distributed Tracing Platform

    Project mention: How To Use OpenTelemetry With AWS Lambda | dev.to | 2021-11-11

    In production, you would probably want to have this sent to some other tool like Jaeger or an observability vendor.

  • Scout APM

    Scout APM: A developer's best friend. Try free for 14-days. Scout APM uses tracing logic that ties bottlenecks to source code so you know the exact line of code causing performance issues and can get back to building a great product faster.

  • GitHub repo vitess

    Vitess is a database clustering system for horizontal scaling of MySQL.

    Project mention: Transition to FAANG Interviews | reddit.com/r/ExperiencedDevs | 2021-11-28

    GitHub pretty much did that. They use Vitess to scale MySQL.

  • GitHub repo go-sql-driver/mysql

    Go MySQL Driver is a MySQL driver for Go's (golang) database/sql package (by go-sql-driver)

    Project mention: Which SQL driver to use with Postgres | reddit.com/r/golang | 2021-11-27

    I'm following Alex Edwards web dev book, he uses MySQL with go-sql-driver/mysql , i want to use Postgres, which one should I use?

  • GitHub repo groupcache

    groupcache is a caching and cache-filling library, intended as a replacement for memcached in many cases.

    Project mention: go-generics-cache: An in-memory key:value store/cache library for Go Generics | reddit.com/r/golang | 2021-11-16

    https://github.com/golang/groupcache is managing distributed caching that addresses thundering herd problem of memcache.

  • GitHub repo badger

    Fast key-value DB in Go.

    Project mention: What is "the go way" for File-based cache in multiple plattforms (UNIX and Windows mainly)? | reddit.com/r/golang | 2021-09-29

    I’m pretty sure Badger is cross platform. It’s embedded, so I don’t see any reason why not.

  • GitHub repo dolt

    Dolt – It's Git for Data

    Project mention: Dolt Is Git for Data | news.ycombinator.com | 2021-09-27
  • GitHub repo rqlite

    The lightweight, distributed relational database built on SQLite

    Project mention: Cloudflare Durable Objects Are Now Generally Available | news.ycombinator.com | 2021-11-15
  • GitHub repo TinyGo

    Go compiler for small places. Microcontrollers, WebAssembly, and command-line tools. Based on LLVM.

    Project mention: tinygo pico serial output with println | reddit.com/r/raspberrypipico | 2021-11-23
  • GitHub repo milvus

    An open-source vector database for embedding similarity search and AI applications.

    Project mention: [P] New open-source vector search solution | reddit.com/r/MachineLearning | 2021-11-16

    How is it compare to https://milvus.io/ ?

  • GitHub repo Tile38

    Real-time Geospatial and Geofencing

    Project mention: Tile38 - a geolocation data store, spatial index, and realtime geofence | reddit.com/r/golang | 2021-08-14
  • GitHub repo migrate

    Database migrations. CLI and Golang library.

    Project mention: Are entity framework tools typically avoided with MySQL & Go and are there alternatives for migration script tooling that version control the entire schema like SSDT? | reddit.com/r/golang | 2021-11-16

    Not that I know of. Generally the approach taken is to have ordered migration scripts and a schema version, and then have code automatically apply the appropriate set of migrations in the right order, for example using golang-migrate.

  • GitHub repo pgweb

    Cross-platform client for PostgreSQL databases

    Project mention: A nice front end for reading mysql or postgres data? | reddit.com/r/selfhosted | 2021-10-17

    For postgresql, pgweb is perfect.

  • GitHub repo kingshard

    A high-performance MySQL proxy

  • GitHub repo go-cache

    An in-memory key:value store/cache (similar to Memcached) library for Go, suitable for single-machine applications.

    Project mention: Cache locally using text file | reddit.com/r/learnprogramming | 2021-07-28

    implementing runtime cache using map seems doable, i may just learn from github.com/patrickmn/go-cache but i dont understand what does it mean cache locally using text file. does it mean I have to:

  • GitHub repo VictoriaMetrics

    VictoriaMetrics: fast, cost-effective monitoring solution and time series database

    Project mention: Prometheus Agent Mode, for Metric Forwarding | news.ycombinator.com | 2021-11-18

    There were a bunch of alternatives to solve this before, like grafana agent [0], vm agent [1] or opentelemetry collector [2].

    [0] - https://github.com/grafana/agent

    [1] - https://github.com/VictoriaMetrics/VictoriaMetrics/tree/mast...

    [2] - https://github.com/open-telemetry/opentelemetry-collector

  • GitHub repo BigCache

    Efficient cache for gigabytes of data written in Go.

  • GitHub repo bbolt

    An embedded key/value database for Go.

    Project mention: How should I name my package repository, when there are multiple implementations in other languages in my github account? | reddit.com/r/golang | 2021-11-27

    There is also an option to have a custom domain like go.etcd.io/bbolt

NOTE: The open source projects on this list are ordered by number of github stars. The number of mentions indicates repo mentiontions in the last 12 Months or since we started tracking (Dec 2020). The latest post mention was on 2021-11-29.

Go Database related posts

Index

What are some of the best open-source Database projects in Go? This list will help you:

Project Stars
1 prometheus 39,746
2 etcd 37,977
3 tidb 29,655
4 cockroach 22,535
5 InfluxDB 22,460
6 dgraph 16,955
7 jaeger 14,674
8 vitess 12,906
9 go-sql-driver/mysql 11,674
10 groupcache 10,871
11 badger 10,018
12 dolt 9,758
13 rqlite 9,071
14 TinyGo 8,933
15 milvus 8,818
16 Tile38 7,855
17 migrate 7,585
18 pgweb 7,096
19 kingshard 5,866
20 go-cache 5,568
21 VictoriaMetrics 5,447
22 BigCache 5,247
23 bbolt 4,942
Find remote jobs at our new job board 99remotejobs.com. There are 34 new remote jobs listed recently.
Are you hiring? Post a new remote job listing for free.
SaaSHub - Software Alternatives and Reviews
SaaSHub helps you find the best software and product alternatives
www.saashub.com