Top 21 Go Sqlite Projects
-
I've always thought it interesting that there was a time when large(ish) websites were hosted using servers that would struggle to outperform a modern smart toaster or wristwatch, and yet modern web applications tend to demand a dramatic distributed architecture. I like the examples in this article showing what a single modern server can do when you're not scaling to Google's level.
As an aside, what about distributed derivatives of sqlite, like rqlite, as a response to the criticism that sqlite requires your database server to also be your web server. Could something like rqlite also provide a way for an sqlite database to grow into a distributed cluster at a later point?
-
Makefile requires migrate - a migration tool: https://github.com/golang-migrate/migrate. I stated it in Makefile and in README, but it's probably not very obvious: Requirements: migrate Migration tool: https://github.com/golang-migrate/migrate
-
SonarQube
Static code analysis for 29 languages.. Your projects are multi-language. So is SonarQube analysis. Find Bugs, Vulnerabilities, Security Hotspots, and Code Smells so you can release quality code every time. Get started analyzing your projects today for free.
-
xo
Command line tool to generate idiomatic Go code for SQL databases supporting PostgreSQL, MySQL, SQLite, Oracle, and Microsoft SQL Server (by xo)
I'm a big fan of the database first code generator approach to talking to an SQL database, so much so that I wrote pggen[1] (not to be confused with pggen[2], as far as I can tell a sqlc fork, which I just recently learned about).
I'm a really big partisan of this approach, but I think I'd like to play the devil's advocate here and lay out some of the weaknesses of both a database first approach in general and sqlc in particular.
All database first approaches struggle with SQL metaprogramming when compared with a query builder library or an ORM. For the most part, this isn't an issue. Just writing SQL and using parameters correctly can get you very far, but there are a few times when you really need it. In particular, faceted search and pagination are both most naturally expressed via runtime metaprogramming of the SQL queries that you want to execute.
Another drawback is poor support from the database for this kind of approach. I only really know how postgres does here, and I'm not sure how well other databases expose their queries. When writing one of these tools you have to resort to tricks like creating temporary views in order infer the argument and return types of a query. This is mostly opaque to the user, but results in weird stuff bubbling up to the API like the tool not being able to infer nullability of arguments and return values well and not being able to support stuff like RETURNING in statements. sqlc is pretty brilliant because it works around this by reimplementing the whole parser and type checker for postgres in go, which is awesome, but also a lot of work to maintain and potentially subtlety wrong.
A minor drawback is that you have to retrain your users to write `x = ANY($1)` instead of `x IN ?`. Most ORMs and query builders seem to lean on their metaprogramming abilities to auto-convert array arguments in the host language into tuples. This is terrible and makes it really annoying when you want to actually pass an array into a query with an ORM/query builder, but it's the convention that everyone is used to.
There are some other issues that most of these tools seem to get wrong, but are not impossible in principle to deal with for a database first code generator. The biggest one is correct handling of migrations. Most of these tools, sqlc included, spit out the straight line "obvious" go code that most people would write to scan some data out of a db. They make a struct, then pass each of the field into Scan by reference to get filled in. This works great until you have a query like `SELECT * FROM foos WHERE field = $1` and then run `ALTER TABLE foos ADD COLUMN new_field text`. Now the deployed server is broken and you need to redeploy really fast as soon as you've run migrations. opendoor/pggen handles this, but I'm not aware of other database first code generators that do (though I could definitely have missed one).
Also the article is missing a few more tools in this space. https://github.com/xo/xo. https://github.com/gnormal/gnorm.
-
upper.io/db
Data access layer for PostgreSQL, CockroachDB, MySQL, SQLite and MongoDB with ORM-like features.
upper.io is a viable alternative to GORM. Just a suggestion.
-
mergestat
Query git repositories with SQL. Generate reports, perform status checks, analyze codebases. 🔍 📊
Project mention: Askgit: Command-line tool for running SQL queries on Git repositories | reddit.com/r/CKsTechNews | 2021-11-27 -
Project mention: Why Google Treats SQL Like Code and You Should Too | news.ycombinator.com | 2022-01-13
Yep, something like Flyway or Liquibase in the Java world has always made a lot of sense, though you also should be able to explicitly say when you do/don't want to run said migrations (e.g. run different profiles).
Of course, recently i've also found Dbmate (https://github.com/amacneil/dbmate) which is a lightweight and language-agnostic tool that supports a variety of DBMSes (PostgreSQL, MySQL/MariaDB, SQLite and more recently even ClickHouse) and can be used for running SQL migration scripts regardless of what technologies that particular app uses (Java, .NET, Node, PHP and so on).
Now, setting up the CI might be a little bit more difficult, but having one tool across all of the services in your architecture can be worth it!
-
https://github.com/pressly/goose. This migration tool is close to alembic.
-
Scout APM
Less time debugging, more time building. Scout APM allows you to find and fix performance issues with no hassle. Now with error monitoring and external services monitoring, Scout is a developer's best friend when it comes to application development.
-
https://github.com/doug-martin/goqu for building SQL queries. Supports MySQL and Postgres at least - super handy!
-
Project mention: Graphviz: Open-source graph visualization software | news.ycombinator.com | 2022-01-17
Autogenerated database documentation is often pretty hit and miss but tbls[1] does a pretty good job in that space. Especially when you comment on your tables, fields, views, functions etc (which is a good habit anyway!) the output is quite useful
-
github.com/gobuffalo/pop: CLI (DSL + Explicit SQL), and
-
From the schema, a TypeScript Prisma Client can be generated that can be used in Node.js applications - including Next.js! A Go Prisma Client is also in the works.
-
-
Jet has full support for SQLite. https://github.com/go-jet/jet
-
-
Project mention: Containerize Go and SQLite with Docker – 9MB Image Size | news.ycombinator.com | 2022-01-17
> C libraries are required to interact with SQLite
Or: modernc.org/sqlite (https://github.com/zombiezen/go-sqlite), "an automatically generated translation of the original C source code of SQLite into Go"
-
-
-
Project mention: I just found a decent, yet underrated, interactive CLI for MySQL, PostgreSQL, and SQLite | reddit.com/r/Database | 2021-06-28
-
dbbench
🏋️ dbbench is a simple database benchmarking tool which supports several databases and own scripts
-
Project mention: CLI tool to import/export csv/xlsx files and run SQL queries | reddit.com/r/SideProject | 2021-06-13
-
Project mention: Embedded write-heavy on-disk cache, write-amplification | reddit.com/r/golang | 2022-01-06
I'm looking for an embedded database/KV-store that supports a write-heavy workload of large blocks of bytes and some kind of eviction policy. I'm currently using sqlite3 with a bunch of triggers and the blob API, but it's not really suitable for write-heavy workloads. I've currently exposed the interface somewhat in https://github.com/anacrolix/squirrel, the primary use case is from https://github.com/anacrolix/torrent. My recent research suggests an LSM-based KV-store like rocksdb or leveldb, but those don't have great interfaces in Go, and don't seem to support an eviction policy as far as I can tell (which is surprising given they would be very well suited to it). There are some alternatives like buntdb, but those all look designed for smaller/string values.
Go Sqlite related posts
- Containerize Go and SQLite with Docker – 9MB Image Size
- Consider SQLite
- Library for sqlite3 recommendations?
- Distributed SQLite for Rust
- Askgit: Command-line tool for running SQL queries on Git repositories
- Askgit: Command-line tool for running SQL queries on Git repositories
- Securing your application credentials and other sensitive keys
Index
What are some of the best open-source Sqlite projects in Go? This list will help you:
| Project | Stars | |
|---|---|---|
| 1 | rqlite | 9,317 |
| 2 | migrate | 7,884 |
| 3 | xo | 2,989 |
| 4 | upper.io/db | 2,872 |
| 5 | mergestat | 2,738 |
| 6 | dbmate | 2,282 |
| 7 | goose | 2,172 |
| 8 | goqu | 1,371 |
| 9 | tbls | 1,243 |
| 10 | gobuffalo/pop | 1,158 |
| 11 | prisma-client-go | 1,101 |
| 12 | go-sqlbuilder | 692 |
| 13 | jet | 607 |
| 14 | dbhub.io | 198 |
| 15 | go-sqlite | 160 |
| 16 | cachego | 159 |
| 17 | Sqinn-Go | 94 |
| 18 | dbui | 71 |
| 19 | dbbench | 62 |
| 20 | csv-sql | 24 |
| 21 | squirrel | 12 |
Are you hiring? Post a new remote job listing for free.
