Show HN: My Go SQLite driver did poorly on a benchmark, so I fixed it

• go-sqlite3

  21 265 9.5 Go

  Go bindings to SQLite using wazero (by ncruces)

https://github.com/ncruces/go-sqlite3/compare/0b093b7c...986...

Creating good, representative, benchmarks is hard. I use speedtest1 [1] but this tests WASM/SQLite and my VFSes, not the Go driver that wraps it.

I already forked this benchmark and will run it regularly, but I may also adapt it into a Go benchmark (easier to run, benchstat compatible, etc).

[1] https://sqlite.org/src/file/test/speedtest1.c

• wazero

  52 4,550 9.8 Go

  wazero: the zero dependency WebAssembly runtime for Go developers

  

• InfluxDB

  www.influxdata.com sponsored

  Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

  

• go-sqlite-bench

  1 20 7.1 Go

  Benchmarks for Golang SQLite Drivers (by ncruces)

• Sqinn-Go

  8 397 5.5 Go

  Golang SQLite without cgo

Note that Squinn is not "native go" or "go only", but the owner insists on misleading people:

https://github.com/cvilsmeier/sqinn-go/issues/8

• sqinn

  4 74 7.0 C

  SQLite over stdin/stdout

First part of the README that hasn't changed in 2 months:

> Sqinn-Go is a Go (Golang) library for accessing SQLite databases without cgo. It uses Sqinn https://github.com/cvilsmeier/sqinn under the hood. It starts Sqinn as a child process (os/exec) and communicates with Sqinn over stdin/stdout/stderr. The Sqinn child process then does the SQLite work.

> If you want SQLite but do not want cgo, Sqinn-Go can be a solution.

This seems pretty clear to me about what's happening. It makes an OS call to a third-party executable (sqinn), pipes the result from stdout back into the Go code. The advantage is you don't have to compile the C code alongside your Go code.

Honestly, I don't really know how much more clear the author could be. I guess if you don't know what stdin, stdout, and stderr are it might be confusing? It's hard to imagine that a programmer who is interested in this library isn't familiar with those concepts though.

• sqlite

  72 - -

> I would've probably picked the modernc variation

Heads up about the modernc library, it has been stuck on an old version of sqlite for several months [1]. It seems like maintainer time is the limiting factor [2]. There has been a call to arms on that issue page, the maintainer is looking for help, but it looks like not much has arrived. It seems like it might trace back to blockers in the C-to-Go compiler.

It's a major undertaking and a very impressive piece of work, but I'm not surprised it's a struggle when big roadblocks get hit. I hope they find a way to progress, but I'm very relieved to be seeing some CGo-free alternatives like ncruces/go-sqlite3 emerging. I'm going to give it a try for sure and see if I can live with the compromises.

Squinn-go looks very compelling too, but I don't like that it requires the squinn binary to already be installed on a user's machine, I think that gives with one hand and takes with the other: sure, I get to avoid CGo, but I also lose the turnkey, single-command install, static build benefits Go brings out of the box.

Seconding the point about nitty gritty, I'd read it for sure too!

  [1]: https://gitlab.com/cznic/sqlite/-/issues/154

• libsql

  21 7,720 9.9 C

  libSQL is a fork of SQLite that is both Open Source, and Open Contributions.

  

A bit of a tangent but for those who’d like to use SQLite for a backend, running it as a separate daemon could be an interesting choice, which would also remove there need of Cgo for the build and maybe make things like separate background job processes easier to accomplish. See [1], [2].

—-

1: https://github.com/tursodatabase/libsql/tree/main/libsql-ser...

2: https://news.ycombinator.com/item?id=38602175

• WorkOS

  workos.com sponsored

  The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.

  

• sqlnotebook

  1 588 7.6 C#

  Casual data exploration in SQL

The meat of the P/Invoke code is in here: https://github.com/electroly/sqlnotebook/tree/master/src/Sql...

The parent directory includes code that uses it. I'm most proud of this SQLite virtual table module that proxies queries to remote ADO.NET connections, allowing you to write joins directly between local SQLite tables and remote SQL Server tables. https://github.com/electroly/sqlnotebook/blob/master/src/Sql...

I've also got a generic virtual table module that lets me easily write table-valued functions in C#: https://github.com/electroly/sqlnotebook/blob/master/src/Sql...

The goal is to provide various "supercharged" features to base SQLite by taking advantage of all the extension points I can. I wish some went further; in particular the virtual table API doesn't "push down" enough of the original query to allow the module to avoid doing O(N+1) queries in some cases.

Suggest a related project