Go SQL

Open-source Go projects categorized as SQL | Edit details

Top 23 Go SQL Projects

  • GitHub repo tidb

    TiDB is an open source distributed HTAP database compatible with the MySQL protocol

    Project mention: Comparing Nginx Performance in Bare Metal and Virtual Environments | news.ycombinator.com | 2021-10-29

    I do agree with you in that regard, however, that's also a dangerous line of thinking.

    There are attempts to provide horizontal scalability for RDBMSes in a transparent way, like TiDB https://pingcap.com/ (which is compatible with the MySQL 5.7 drivers), however, the list of functionality that's sacrificed to achieve easily extensible clusters is a long one: https://docs.pingcap.com/tidb/stable/mysql-compatibility

    There are other technologies, like MongoDB, which sometimes are more successful at a clustered configuration, however most of the traditional RDBMSes work best in a leader-follower type of replication scenario, because even those aforementioned systems oftentimes have data consistency issues that may eventually pop up.

    Essentially, my argument is that the lack of good horizontally scalable databases or other data storage solutions is easily explainable by the fact that the problem itself isn't solvable in any easy way, apart from adopting eventual consistency, which is probably going to create more problems than it will solve in case of any pre-existing code that makes assumptions about what ways it'll be able to access data and operate on it: https://en.wikipedia.org/wiki/Fallacies_of_distributed_compu...

    To that end, i'd perhaps like to suggest an alternative: use a single vertically scalable RDBMS instance when possible, with a hot standby if you have the resources for that. Let the architecture around it be horizontally scalable instead, and let it deal with the complexities of balancing the load and dealing with backpressure - introduce a message queue if you must, maybe even an in-memory one for simplicity's sake, or consider an event based architecture where "what needs to be done" is encapsulated within a data structure that can be passed around and applied whenever possible. In my eyes, such solutions can in many cases be better than losing the many benefits of having a single source of truth.

    Alternatively, consider sharding as a possibility, or, alternatively, do some domain driven design, figure out where to draw some boundaries and split your service into multiple ones that cover the domain with which you need to work with. Then you have one DB for sales, one for account management, one for reports and so on, all separated by something as simple as REST interfaces and with rate limits or any of the other mechanisms.

    If, however, neither of those two groups of approaches don't seem to be suitable for the loads that you're dealing with, then you probably have a team of very smart people and a large amount of resources to figure out what will work best.

    To sum up, if there are no good solutions in the space, perhaps that's because the problems themselves haven't been solved yet. Thus, sooner or later, they'll need to be sidestepped and their impact mitigated in whatever capacity is possible.

  • GitHub repo cockroach

    CockroachDB - the open source, cloud-native distributed SQL database.

    Project mention: Stacked changes: how FB and Google engineers stay unblocked and ship faster | news.ycombinator.com | 2021-11-17

    I'm surprised Reviewable[0] hasn't come up in this discussion. It does a great job of allowing stacked code reviews and even handles rebases nicely; the reviewer sees the diff between commit #1 and commit #1' (prime = after rebase).

    CockroachDB[1] has been using it since very early in the project.

    [0] https://reviewable.io/

    [1] https://github.com/cockroachdb/cockroach

  • Scout APM

    Scout APM: A developer's best friend. Try free for 14-days. Scout APM uses tracing logic that ties bottlenecks to source code so you know the exact line of code causing performance issues and can get back to building a great product faster.

  • GitHub repo go-sql-driver/mysql

    Go MySQL Driver is a MySQL driver for Go's (golang) database/sql package (by go-sql-driver)

    Project mention: Which SQL driver to use with Postgres | reddit.com/r/golang | 2021-11-27

    I'm following Alex Edwards web dev book, he uses MySQL with go-sql-driver/mysql , i want to use Postgres, which one should I use?

  • GitHub repo dolt

    Dolt – It's Git for Data

    Project mention: Dolt Is Git for Data | news.ycombinator.com | 2021-09-27
  • GitHub repo rqlite

    The lightweight, distributed relational database built on SQLite

    Project mention: Cloudflare Durable Objects Are Now Generally Available | news.ycombinator.com | 2021-11-15
  • GitHub repo migrate

    Database migrations. CLI and Golang library.

    Project mention: Are entity framework tools typically avoided with MySQL & Go and are there alternatives for migration script tooling that version control the entire schema like SSDT? | reddit.com/r/golang | 2021-11-16

    Not that I know of. Generally the approach taken is to have ordered migration scripts and a schema version, and then have code automatically apply the appropriate set of migrations in the right order, for example using golang-migrate.

  • GitHub repo usql

    Universal command-line interface for SQL databases

    Project mention: usql v0.9.4 | reddit.com/r/golang | 2021-08-29
  • Nanos

    Run Linux Software Faster and Safer than Linux with Unikernels.

  • GitHub repo go-pg

    Golang ORM with focus on PostgreSQL features and performance

    Project mention: Go for Backend development | reddit.com/r/golang | 2021-09-06
  • GitHub repo go-clean-arch

    Go (Golang) Clean Architecture based on Reading Uncle Bob's Clean Architecture

    Project mention: Any suggestions for a beginner to build a microservice using Go with ES? | reddit.com/r/elasticsearch | 2021-08-11

    Has anyone come across any Golang repo like go-clean-arch which uses elasticsearch? As I am a beginner and wanted to have a bit of practice of building microservice using Go with Elasticsearch.

  • GitHub repo sqlc

    Generate type safe Go from SQL

    Project mention: Which SQL driver to use with Postgres | reddit.com/r/golang | 2021-11-27

    I can recommend pairing it with sqlc: https://github.com/kyleconroy/sqlc

  • GitHub repo go-sqlmock

    Sql mock driver for golang to test database interactions

    Project mention: What are the REST API reference projects that can be used as a guide in 2021? | reddit.com/r/golang | 2021-11-24

    However your point is valid. Check matryer/is for unit test writing. DATA-DOG/go-sqlmock to test database interactions.

  • GitHub repo xo

    Command line tool to generate idiomatic Go code for SQL databases supporting PostgreSQL, MySQL, SQLite, Oracle, and Microsoft SQL Server (by xo)

    Project mention: We Went All in on Sqlc/Pgx for Postgres and Go | news.ycombinator.com | 2021-09-08

    I'm a big fan of the database first code generator approach to talking to an SQL database, so much so that I wrote pggen[1] (not to be confused with pggen[2], as far as I can tell a sqlc fork, which I just recently learned about).

    I'm a really big partisan of this approach, but I think I'd like to play the devil's advocate here and lay out some of the weaknesses of both a database first approach in general and sqlc in particular.

    All database first approaches struggle with SQL metaprogramming when compared with a query builder library or an ORM. For the most part, this isn't an issue. Just writing SQL and using parameters correctly can get you very far, but there are a few times when you really need it. In particular, faceted search and pagination are both most naturally expressed via runtime metaprogramming of the SQL queries that you want to execute.

    Another drawback is poor support from the database for this kind of approach. I only really know how postgres does here, and I'm not sure how well other databases expose their queries. When writing one of these tools you have to resort to tricks like creating temporary views in order infer the argument and return types of a query. This is mostly opaque to the user, but results in weird stuff bubbling up to the API like the tool not being able to infer nullability of arguments and return values well and not being able to support stuff like RETURNING in statements. sqlc is pretty brilliant because it works around this by reimplementing the whole parser and type checker for postgres in go, which is awesome, but also a lot of work to maintain and potentially subtlety wrong.

    A minor drawback is that you have to retrain your users to write `x = ANY($1)` instead of `x IN ?`. Most ORMs and query builders seem to lean on their metaprogramming abilities to auto-convert array arguments in the host language into tuples. This is terrible and makes it really annoying when you want to actually pass an array into a query with an ORM/query builder, but it's the convention that everyone is used to.

    There are some other issues that most of these tools seem to get wrong, but are not impossible in principle to deal with for a database first code generator. The biggest one is correct handling of migrations. Most of these tools, sqlc included, spit out the straight line "obvious" go code that most people would write to scan some data out of a db. They make a struct, then pass each of the field into Scan by reference to get filled in. This works great until you have a query like `SELECT * FROM foos WHERE field = $1` and then run `ALTER TABLE foos ADD COLUMN new_field text`. Now the deployed server is broken and you need to redeploy really fast as soon as you've run migrations. opendoor/pggen handles this, but I'm not aware of other database first code generators that do (though I could definitely have missed one).

    Also the article is missing a few more tools in this space. https://github.com/xo/xo. https://github.com/gnormal/gnorm.

    [1]: https://github.com/opendoor/pggen

  • GitHub repo upper.io/db

    Data access layer for PostgreSQL, CockroachDB, MySQL, SQLite and MongoDB with ORM-like features.

    Project mention: Migrating from PHP to Go | reddit.com/r/golang | 2021-09-30

    upper.io is a viable alternative to GORM. Just a suggestion.

  • GitHub repo askgit

    Query git repositories with SQL. Generate reports, perform status checks, analyze codebases. 🔍 📊

    Project mention: Askgit: Command-line tool for running SQL queries on Git repositories | reddit.com/r/CKsTechNews | 2021-11-27
  • GitHub repo octosql

    OctoSQL is a query tool that allows you to join, analyse and transform data from multiple databases and file formats using SQL.

    Project mention: OctoSQL | reddit.com/r/devopskhan | 2021-10-31
  • GitHub repo sql-migrate

    SQL schema migration tool for Go.

    Project mention: How do you install commands using go.mod | reddit.com/r/golang | 2021-11-02

    There are some packages in my project that are not used in the source code, but they're used as commands (i.e. https://github.com/vektra/mockery https://github.com/rubenv/sql-migrate).

  • GitHub repo goose

    A database migration tool. Supports SQL migrations and Go functions.

    Project mention: Dockerize & Backup A Postgres Database | dev.to | 2021-11-16

    Great, so our data has been backed up! This means that you can restore folks back to a particular point in time when things were as they should be. So long as you have an application that can reliably serve this data (ruminates on articles using Goose to accomplish database versioning) you can restore functionality and value to your users.

  • GitHub repo cloudquery

    The open-source cloud asset inventory powered by SQL.

    Project mention: Running CloudQuery in AWS Lambda | dev.to | 2021-11-29

    git clone https://github.com/cloudquery/cloudquery.git

  • GitHub repo graphjin

    GraphJin - Build APIs in 5 minutes with GraphQL. An instant GraphQL to SQL compiler.

    Project mention: Go Masterpieces | reddit.com/r/golang | 2021-11-16

    Graphjin a GraphQL to SQL compiler https://github.com/dosco/graphjin

  • GitHub repo xorm

    xorm是一个简单而强大的Go语言ORM库,通过它可以使数据库操作非常简便。本库是基于原版xorm的定制增强版本,为xorm提供类似ibatis的配置文件及动态SQL支持,支持AcitveRecord操作 (by xormplus)

  • GitHub repo goqu

    SQL builder and query library for golang

    Project mention: Migrating from PHP to Go | reddit.com/r/golang | 2021-09-30

    https://github.com/doug-martin/goqu for building SQL queries. Supports MySQL and Postgres at least - super handy!

  • GitHub repo null

    reasonable handling of nullable values (by guregu)

    Project mention: Gonion - Golang Client for querying Tor network data | reddit.com/r/golang | 2021-09-29

    Unfortunately in Go, switching to *bool makes the api a little more awkward to use since users that need to set true or false have to define a local variable then use a pointer to that. Another option would be something like null, but that adds a dependency to your currently-dependency-free project. If anyone has a better solution to this pattern, I'd love to hear it.

  • GitHub repo CovenantSQL

    A decentralized, trusted, high performance, SQL database with blockchain features

NOTE: The open source projects on this list are ordered by number of github stars. The number of mentions indicates repo mentiontions in the last 12 Months or since we started tracking (Dec 2020). The latest post mention was on 2021-11-29.

Go SQL related posts

Index

What are some of the best open-source SQL projects in Go? This list will help you:

Project Stars
1 tidb 29,655
2 cockroach 22,535
3 go-sql-driver/mysql 11,674
4 dolt 9,758
5 rqlite 9,071
6 migrate 7,585
7 usql 6,845
8 go-pg 4,883
9 go-clean-arch 4,721
10 sqlc 4,289
11 go-sqlmock 3,939
12 xo 2,949
13 upper.io/db 2,803
14 askgit 2,680
15 octosql 2,560
16 sql-migrate 2,313
17 goose 2,002
18 cloudquery 1,990
19 graphjin 1,599
20 xorm 1,469
21 goqu 1,319
22 null 1,295
23 CovenantSQL 1,236
Find remote jobs at our new job board 99remotejobs.com. There are 34 new remote jobs listed recently.
Are you hiring? Post a new remote job listing for free.
SaaSHub - Software Alternatives and Reviews
SaaSHub helps you find the best software and product alternatives
www.saashub.com