Top 23 Go SQL Projects
TiDB is an open source distributed HTAP database compatible with the MySQL protocolProject mention: Comparing Nginx Performance in Bare Metal and Virtual Environments | news.ycombinator.com | 2021-10-29
I do agree with you in that regard, however, that's also a dangerous line of thinking.
There are attempts to provide horizontal scalability for RDBMSes in a transparent way, like TiDB https://pingcap.com/ (which is compatible with the MySQL 5.7 drivers), however, the list of functionality that's sacrificed to achieve easily extensible clusters is a long one: https://docs.pingcap.com/tidb/stable/mysql-compatibility
There are other technologies, like MongoDB, which sometimes are more successful at a clustered configuration, however most of the traditional RDBMSes work best in a leader-follower type of replication scenario, because even those aforementioned systems oftentimes have data consistency issues that may eventually pop up.
Essentially, my argument is that the lack of good horizontally scalable databases or other data storage solutions is easily explainable by the fact that the problem itself isn't solvable in any easy way, apart from adopting eventual consistency, which is probably going to create more problems than it will solve in case of any pre-existing code that makes assumptions about what ways it'll be able to access data and operate on it: https://en.wikipedia.org/wiki/Fallacies_of_distributed_compu...
To that end, i'd perhaps like to suggest an alternative: use a single vertically scalable RDBMS instance when possible, with a hot standby if you have the resources for that. Let the architecture around it be horizontally scalable instead, and let it deal with the complexities of balancing the load and dealing with backpressure - introduce a message queue if you must, maybe even an in-memory one for simplicity's sake, or consider an event based architecture where "what needs to be done" is encapsulated within a data structure that can be passed around and applied whenever possible. In my eyes, such solutions can in many cases be better than losing the many benefits of having a single source of truth.
Alternatively, consider sharding as a possibility, or, alternatively, do some domain driven design, figure out where to draw some boundaries and split your service into multiple ones that cover the domain with which you need to work with. Then you have one DB for sales, one for account management, one for reports and so on, all separated by something as simple as REST interfaces and with rate limits or any of the other mechanisms.
If, however, neither of those two groups of approaches don't seem to be suitable for the loads that you're dealing with, then you probably have a team of very smart people and a large amount of resources to figure out what will work best.
To sum up, if there are no good solutions in the space, perhaps that's because the problems themselves haven't been solved yet. Thus, sooner or later, they'll need to be sidestepped and their impact mitigated in whatever capacity is possible.
CockroachDB - the open source, cloud-native distributed SQL database.Project mention: Stacked changes: how FB and Google engineers stay unblocked and ship faster | news.ycombinator.com | 2021-11-17
I'm surprised Reviewable hasn't come up in this discussion. It does a great job of allowing stacked code reviews and even handles rebases nicely; the reviewer sees the diff between commit #1 and commit #1' (prime = after rebase).
CockroachDB has been using it since very early in the project.
Scout APM: A developer's best friend. Try free for 14-days. Scout APM uses tracing logic that ties bottlenecks to source code so you know the exact line of code causing performance issues and can get back to building a great product faster.
Go MySQL Driver is a MySQL driver for Go's (golang) database/sql package (by go-sql-driver)Project mention: Which SQL driver to use with Postgres | reddit.com/r/golang | 2021-11-27
I'm following Alex Edwards web dev book, he uses MySQL with go-sql-driver/mysql , i want to use Postgres, which one should I use?
Dolt – It's Git for DataProject mention: Dolt Is Git for Data | news.ycombinator.com | 2021-09-27
The lightweight, distributed relational database built on SQLiteProject mention: Cloudflare Durable Objects Are Now Generally Available | news.ycombinator.com | 2021-11-15
Database migrations. CLI and Golang library.Project mention: Are entity framework tools typically avoided with MySQL & Go and are there alternatives for migration script tooling that version control the entire schema like SSDT? | reddit.com/r/golang | 2021-11-16
Not that I know of. Generally the approach taken is to have ordered migration scripts and a schema version, and then have code automatically apply the appropriate set of migrations in the right order, for example using golang-migrate.
Universal command-line interface for SQL databasesProject mention: usql v0.9.4 | reddit.com/r/golang | 2021-08-29
Run Linux Software Faster and Safer than Linux with Unikernels.
Golang ORM with focus on PostgreSQL features and performanceProject mention: Go for Backend development | reddit.com/r/golang | 2021-09-06
Go (Golang) Clean Architecture based on Reading Uncle Bob's Clean ArchitectureProject mention: Any suggestions for a beginner to build a microservice using Go with ES? | reddit.com/r/elasticsearch | 2021-08-11
Has anyone come across any Golang repo like go-clean-arch which uses elasticsearch? As I am a beginner and wanted to have a bit of practice of building microservice using Go with Elasticsearch.
Generate type safe Go from SQLProject mention: Which SQL driver to use with Postgres | reddit.com/r/golang | 2021-11-27
I can recommend pairing it with sqlc: https://github.com/kyleconroy/sqlc
Sql mock driver for golang to test database interactionsProject mention: What are the REST API reference projects that can be used as a guide in 2021? | reddit.com/r/golang | 2021-11-24
However your point is valid. Check matryer/is for unit test writing. DATA-DOG/go-sqlmock to test database interactions.
Command line tool to generate idiomatic Go code for SQL databases supporting PostgreSQL, MySQL, SQLite, Oracle, and Microsoft SQL Server (by xo)Project mention: We Went All in on Sqlc/Pgx for Postgres and Go | news.ycombinator.com | 2021-09-08
I'm a big fan of the database first code generator approach to talking to an SQL database, so much so that I wrote pggen (not to be confused with pggen, as far as I can tell a sqlc fork, which I just recently learned about).
I'm a really big partisan of this approach, but I think I'd like to play the devil's advocate here and lay out some of the weaknesses of both a database first approach in general and sqlc in particular.
All database first approaches struggle with SQL metaprogramming when compared with a query builder library or an ORM. For the most part, this isn't an issue. Just writing SQL and using parameters correctly can get you very far, but there are a few times when you really need it. In particular, faceted search and pagination are both most naturally expressed via runtime metaprogramming of the SQL queries that you want to execute.
Another drawback is poor support from the database for this kind of approach. I only really know how postgres does here, and I'm not sure how well other databases expose their queries. When writing one of these tools you have to resort to tricks like creating temporary views in order infer the argument and return types of a query. This is mostly opaque to the user, but results in weird stuff bubbling up to the API like the tool not being able to infer nullability of arguments and return values well and not being able to support stuff like RETURNING in statements. sqlc is pretty brilliant because it works around this by reimplementing the whole parser and type checker for postgres in go, which is awesome, but also a lot of work to maintain and potentially subtlety wrong.
A minor drawback is that you have to retrain your users to write `x = ANY($1)` instead of `x IN ?`. Most ORMs and query builders seem to lean on their metaprogramming abilities to auto-convert array arguments in the host language into tuples. This is terrible and makes it really annoying when you want to actually pass an array into a query with an ORM/query builder, but it's the convention that everyone is used to.
There are some other issues that most of these tools seem to get wrong, but are not impossible in principle to deal with for a database first code generator. The biggest one is correct handling of migrations. Most of these tools, sqlc included, spit out the straight line "obvious" go code that most people would write to scan some data out of a db. They make a struct, then pass each of the field into Scan by reference to get filled in. This works great until you have a query like `SELECT * FROM foos WHERE field = $1` and then run `ALTER TABLE foos ADD COLUMN new_field text`. Now the deployed server is broken and you need to redeploy really fast as soon as you've run migrations. opendoor/pggen handles this, but I'm not aware of other database first code generators that do (though I could definitely have missed one).
Data access layer for PostgreSQL, CockroachDB, MySQL, SQLite and MongoDB with ORM-like features.Project mention: Migrating from PHP to Go | reddit.com/r/golang | 2021-09-30
upper.io is a viable alternative to GORM. Just a suggestion.
Query git repositories with SQL. Generate reports, perform status checks, analyze codebases. 🔍 📊Project mention: Askgit: Command-line tool for running SQL queries on Git repositories | reddit.com/r/CKsTechNews | 2021-11-27
OctoSQL is a query tool that allows you to join, analyse and transform data from multiple databases and file formats using SQL.Project mention: OctoSQL | reddit.com/r/devopskhan | 2021-10-31
SQL schema migration tool for Go.Project mention: How do you install commands using go.mod | reddit.com/r/golang | 2021-11-02
There are some packages in my project that are not used in the source code, but they're used as commands (i.e. https://github.com/vektra/mockery https://github.com/rubenv/sql-migrate).
A database migration tool. Supports SQL migrations and Go functions.Project mention: Dockerize & Backup A Postgres Database | dev.to | 2021-11-16
Great, so our data has been backed up! This means that you can restore folks back to a particular point in time when things were as they should be. So long as you have an application that can reliably serve this data (ruminates on articles using Goose to accomplish database versioning) you can restore functionality and value to your users.
The open-source cloud asset inventory powered by SQL.Project mention: Running CloudQuery in AWS Lambda | dev.to | 2021-11-29
git clone https://github.com/cloudquery/cloudquery.git
GraphJin - Build APIs in 5 minutes with GraphQL. An instant GraphQL to SQL compiler.Project mention: Go Masterpieces | reddit.com/r/golang | 2021-11-16
Graphjin a GraphQL to SQL compiler https://github.com/dosco/graphjin
xorm是一个简单而强大的Go语言ORM库，通过它可以使数据库操作非常简便。本库是基于原版xorm的定制增强版本，为xorm提供类似ibatis的配置文件及动态SQL支持，支持AcitveRecord操作 (by xormplus)
SQL builder and query library for golangProject mention: Migrating from PHP to Go | reddit.com/r/golang | 2021-09-30
https://github.com/doug-martin/goqu for building SQL queries. Supports MySQL and Postgres at least - super handy!
reasonable handling of nullable values (by guregu)Project mention: Gonion - Golang Client for querying Tor network data | reddit.com/r/golang | 2021-09-29
Unfortunately in Go, switching to *bool makes the api a little more awkward to use since users that need to set true or false have to define a local variable then use a pointer to that. Another option would be something like null, but that adds a dependency to your currently-dependency-free project. If anyone has a better solution to this pattern, I'd love to hear it.
A decentralized, trusted, high performance, SQL database with blockchain features
Go SQL related posts
Stream Processing Database
4 projects | reddit.com/r/Database | 28 Nov 2021
Which SQL driver to use with Postgres
7 projects | reddit.com/r/golang | 27 Nov 2021
Exiting the Vietnam of Programming: Our Journey in Dropping the ORM (in Golang)
1 project | reddit.com/r/golang | 27 Nov 2021
Askgit: Command-line tool for running SQL queries on Git repositories
1 project | reddit.com/r/CKsTechNews | 27 Nov 2021
Askgit: Command-line tool for running SQL queries on Git repositories
1 project | news.ycombinator.com | 27 Nov 2021
Which Go database/storage package do you go for first when your program needs to store a moderate amount of organized data?
4 projects | reddit.com/r/golang | 22 Nov 2021
[HELP]One to many in sqlx.
3 projects | reddit.com/r/golang | 22 Nov 2021
What are some of the best open-source SQL projects in Go? This list will help you:
Are you hiring? Post a new remote job listing for free.