Top 23 Go Database Projects
The Prometheus monitoring system and time series database.Project mention: My media server dashboard, been pretty happy with it for a while, trying to decided if I need any other info on it | reddit.com/r/homelab | 2021-11-29
Are you looking for general container metrics, or the media server-specific metrics OP has? If you're just looking for general metrics, I just made a blog post tutorial on this, I can PM you if interested but I don't want to spam/self promote here. The three basic components are Grafana, Prometheus, and node exporter
Distributed reliable key-value store for the most critical data of a distributed systemProject mention: etcd | reddit.com/r/JavaOnTheEdge | 2021-11-07
Run Linux Software Faster and Safer than Linux with Unikernels.
TiDB is an open source distributed HTAP database compatible with the MySQL protocolProject mention: Comparing Nginx Performance in Bare Metal and Virtual Environments | news.ycombinator.com | 2021-10-29
I do agree with you in that regard, however, that's also a dangerous line of thinking.
There are attempts to provide horizontal scalability for RDBMSes in a transparent way, like TiDB https://pingcap.com/ (which is compatible with the MySQL 5.7 drivers), however, the list of functionality that's sacrificed to achieve easily extensible clusters is a long one: https://docs.pingcap.com/tidb/stable/mysql-compatibility
There are other technologies, like MongoDB, which sometimes are more successful at a clustered configuration, however most of the traditional RDBMSes work best in a leader-follower type of replication scenario, because even those aforementioned systems oftentimes have data consistency issues that may eventually pop up.
Essentially, my argument is that the lack of good horizontally scalable databases or other data storage solutions is easily explainable by the fact that the problem itself isn't solvable in any easy way, apart from adopting eventual consistency, which is probably going to create more problems than it will solve in case of any pre-existing code that makes assumptions about what ways it'll be able to access data and operate on it: https://en.wikipedia.org/wiki/Fallacies_of_distributed_compu...
To that end, i'd perhaps like to suggest an alternative: use a single vertically scalable RDBMS instance when possible, with a hot standby if you have the resources for that. Let the architecture around it be horizontally scalable instead, and let it deal with the complexities of balancing the load and dealing with backpressure - introduce a message queue if you must, maybe even an in-memory one for simplicity's sake, or consider an event based architecture where "what needs to be done" is encapsulated within a data structure that can be passed around and applied whenever possible. In my eyes, such solutions can in many cases be better than losing the many benefits of having a single source of truth.
Alternatively, consider sharding as a possibility, or, alternatively, do some domain driven design, figure out where to draw some boundaries and split your service into multiple ones that cover the domain with which you need to work with. Then you have one DB for sales, one for account management, one for reports and so on, all separated by something as simple as REST interfaces and with rate limits or any of the other mechanisms.
If, however, neither of those two groups of approaches don't seem to be suitable for the loads that you're dealing with, then you probably have a team of very smart people and a large amount of resources to figure out what will work best.
To sum up, if there are no good solutions in the space, perhaps that's because the problems themselves haven't been solved yet. Thus, sooner or later, they'll need to be sidestepped and their impact mitigated in whatever capacity is possible.
CockroachDB - the open source, cloud-native distributed SQL database.Project mention: Stacked changes: how FB and Google engineers stay unblocked and ship faster | news.ycombinator.com | 2021-11-17
I'm surprised Reviewable hasn't come up in this discussion. It does a great job of allowing stacked code reviews and even handles rebases nicely; the reviewer sees the diff between commit #1 and commit #1' (prime = after rebase).
CockroachDB has been using it since very early in the project.
Scalable datastore for metrics, events, and real-time analyticsProject mention: How to stream IoT data into Excel | reddit.com/r/IOT | 2021-11-11
I would suggest using something better suited to IoT data than ... a spreadsheet. I'd recommend looking at one of the Time Series Databases for this. 1) QuestDB or 2) InfluxDB as these are much better suited to streaming data.
Native GraphQL Database with graph backendProject mention: DGraph Data Modeling: Part 1 - Timestamps | dev.to | 2021-11-26
In my opinion, DGraph is the best competition to Firestore there is (and much much more). Imagine Firestore with Subscriptions, plus a Graph Database for complex searching, plus any kind of relational data you can think of, wrapped in a GraphQL interface, or a backend custom language called DQL.
CNCF Jaeger, a Distributed Tracing PlatformProject mention: How To Use OpenTelemetry With AWS Lambda | dev.to | 2021-11-11
In production, you would probably want to have this sent to some other tool like Jaeger or an observability vendor.
Scout APM: A developer's best friend. Try free for 14-days. Scout APM uses tracing logic that ties bottlenecks to source code so you know the exact line of code causing performance issues and can get back to building a great product faster.
Vitess is a database clustering system for horizontal scaling of MySQL.Project mention: Transition to FAANG Interviews | reddit.com/r/ExperiencedDevs | 2021-11-28
GitHub pretty much did that. They use Vitess to scale MySQL.
Go MySQL Driver is a MySQL driver for Go's (golang) database/sql package (by go-sql-driver)Project mention: Which SQL driver to use with Postgres | reddit.com/r/golang | 2021-11-27
I'm following Alex Edwards web dev book, he uses MySQL with go-sql-driver/mysql , i want to use Postgres, which one should I use?
groupcache is a caching and cache-filling library, intended as a replacement for memcached in many cases.Project mention: go-generics-cache: An in-memory key:value store/cache library for Go Generics | reddit.com/r/golang | 2021-11-16
https://github.com/golang/groupcache is managing distributed caching that addresses thundering herd problem of memcache.
Fast key-value DB in Go.Project mention: What is "the go way" for File-based cache in multiple plattforms (UNIX and Windows mainly)? | reddit.com/r/golang | 2021-09-29
I’m pretty sure Badger is cross platform. It’s embedded, so I don’t see any reason why not.
Dolt – It's Git for DataProject mention: Dolt Is Git for Data | news.ycombinator.com | 2021-09-27
The lightweight, distributed relational database built on SQLiteProject mention: Cloudflare Durable Objects Are Now Generally Available | news.ycombinator.com | 2021-11-15
Go compiler for small places. Microcontrollers, WebAssembly, and command-line tools. Based on LLVM.Project mention: tinygo pico serial output with println | reddit.com/r/raspberrypipico | 2021-11-23
An open-source vector database for embedding similarity search and AI applications.Project mention: [P] New open-source vector search solution | reddit.com/r/MachineLearning | 2021-11-16
How is it compare to https://milvus.io/ ?
Real-time Geospatial and GeofencingProject mention: Tile38 - a geolocation data store, spatial index, and realtime geofence | reddit.com/r/golang | 2021-08-14
Database migrations. CLI and Golang library.Project mention: Are entity framework tools typically avoided with MySQL & Go and are there alternatives for migration script tooling that version control the entire schema like SSDT? | reddit.com/r/golang | 2021-11-16
Not that I know of. Generally the approach taken is to have ordered migration scripts and a schema version, and then have code automatically apply the appropriate set of migrations in the right order, for example using golang-migrate.
Cross-platform client for PostgreSQL databasesProject mention: A nice front end for reading mysql or postgres data? | reddit.com/r/selfhosted | 2021-10-17
For postgresql, pgweb is perfect.
A high-performance MySQL proxy
An in-memory key:value store/cache (similar to Memcached) library for Go, suitable for single-machine applications.Project mention: Cache locally using text file | reddit.com/r/learnprogramming | 2021-07-28
implementing runtime cache using map seems doable, i may just learn from github.com/patrickmn/go-cache but i dont understand what does it mean cache locally using text file. does it mean I have to:
VictoriaMetrics: fast, cost-effective monitoring solution and time series databaseProject mention: Prometheus Agent Mode, for Metric Forwarding | news.ycombinator.com | 2021-11-18
There were a bunch of alternatives to solve this before, like grafana agent , vm agent  or opentelemetry collector .
Efficient cache for gigabytes of data written in Go.
An embedded key/value database for Go.Project mention: How should I name my package repository, when there are multiple implementations in other languages in my github account? | reddit.com/r/golang | 2021-11-27
There is also an option to have a custom domain like go.etcd.io/bbolt
Go Database related posts
Transition to FAANG Interviews
1 project | reddit.com/r/ExperiencedDevs | 28 Nov 2021
Stream Processing Database
4 projects | reddit.com/r/Database | 28 Nov 2021
How should I name my package repository, when there are multiple implementations in other languages in my github account?
2 projects | reddit.com/r/golang | 27 Nov 2021
DGraph Data Modeling: Part 1 - Timestamps
2 projects | dev.to | 26 Nov 2021
Which Go database/storage package do you go for first when your program needs to store a moderate amount of organized data?
4 projects | reddit.com/r/golang | 22 Nov 2021
tinygo pico serial output with println
1 project | reddit.com/r/raspberrypipico | 23 Nov 2021
tinygo serial output println
1 project | reddit.com/r/raspberrypipico | 23 Nov 2021
What are some of the best open-source Database projects in Go? This list will help you:
Are you hiring? Post a new remote job listing for free.