go-sqlite3 VS bun

SQLite encryption at-rest is a hot requested feature of both the “default” CGo driver [1] and the transpiled alternative driver [2]. So, this is a feature I wanted to bring to my own Wasm based Go driver/bindings [3].

Open-source SQLite encryption extensions have had a troubled last few years. For whatever reason, in 2020 the (undocumented) feature that made it easy to offer page-level encryption was removed [4]. Some solutions are stuck with SQLite 3.31.1, but Ulrich Telle stepped up with a VFS approach [5].

Still, their solution seemed harder than something I'd want to maintain, as it requires understanding the structure of what's being written to disk at the VFS layer. So, I looked at full disk encryption for something with less of an impedance mismatch.

Specifically, I'm using the Adiantum tweakable and length-preserving encryption (with 4K blocks, matching the default SQLite page size), and encrypting whole files (rather than page content).

I'm not a cryptographer, so I'd really appreciate some roasting before release.

There is nothing very Go specific about this (apart from the implementation) so if there are no obvious flaws, it may make sense to port it to C/Rust/etc and make it a loadable extension.

[1] https://github.com/mattn/go-sqlite3/pull/1109

for what it's worth, the two pool approach is suggested here by a collaborator to github.com/mattn/go-sqlite3: https://github.com/mattn/go-sqlite3/issues/1179#issuecomment...

SQLite is great. I've also recently settled on it as a key-value store, after considering a few purpose-built key-value solutions. Turns out that it's really easy to make SQLite work as a key-value store, but very difficult to make key-value stores relational.

Just be careful with `:memory:` databases. From the mattn/go-sqlite3 FAQ[1]:

> Each connection to ":memory:" opens a brand new in-memory sql database, so if the stdlib's sql engine happens to open another connection and you've only specified ":memory:", that connection will see a brand new database. A workaround is to use "file::memory:?cache=shared" (or "file:foobar?mode=memory&cache=shared"). Every connection to this string will point to the same in-memory database.

I noticed strange behaviors with just `:memory:` where tables would just disappear at random, and this workaround helped. Make sure to use a unique filename as the `file:` value, especially if using this in tests.

[1]: https://github.com/mattn/go-sqlite3#faq

github.com/mattn/go-sqlite3

For building my web server, I chose to use the Gin framework as the foundation of my app. It was incredibly easy to understand and work with, and I was pleasantly surprised by how seamlessly it integrated with writing unit tests for the server. To handle the database, I leveraged the power of go-sqlite and migrate for efficient SQL queries and migrations. These libraries proved to be both powerful and user-friendly, making the development process a breeze.

https://github.com/mattn/go-sqlite3/blob/master/_example/sim...

SQLite does have an optional "user authentication" extension, though I've not personally tried it out:

https://www.sqlite.org/src/doc/trunk/ext/userauth/user-auth....

The widely used Go SQLite library by mattn says it supports it, if that's useful:

https://github.com/mattn/go-sqlite3#user-authentication

I have an OSS project, sq which is a data-wrangling swiss-army knife for structured data. Think of it as jq for databases. It supports Postgres, SQLServer, MySQL and - relevantly - SQLite. It embeds SQLite via CGo and the mattn/go-sqlite3 driver.

IDE: use whatever make you productive. I personally use vscode. VCS: git, as golang communities use github heavily as base for many libraries. AFAIK Linter: use staticcheck for linting as it looks like mostly used linting tool in go, supported by many also. In Vscode it will be recommended once you install go plugin. Libraries/Framework: actually the standard libraries already included many things you need, decent enough for your day-to-day development cycles(e.g. `net/http`). But here are things for extra: - Struct fields validator: validator - Http server lib: chi router , httprouter , fasthttp (for non standard http implementations, but fast) - Web Framework: echo , gin , fiber , beego , etc - Http client lib: most already covered by stdlib(net/http), so you rarely need extra lib for this, but if you really need some are: resty - CLI: cobra - Config: godotenv , viper - DB Drivers: sqlx , postgre , sqlite , mysql - nosql: redis , mongodb , elasticsearch - ORM: gorm , entgo , sqlc(codegen) - JS Transpiler: gopherjs - GUI: fyne - grpc: grpc - logging: zerolog - test: testify , gomock , dockertest - and many others you can find here

It has a decent compatibility with both Jest and Vitest's APIs (you can track progress here so you can use it as almost a drop-in replacement for either. Just as Node's, it has describe/it, mock, test and others, but with the expect syntax (which I find more readable). For example:

In this third and final article in the series on HTML Streaming, we will explore the practical implementation of the Diff DOM Streaming library in web browsing. This approach will allow any website using web components to retain its state during browsing. We will discuss in detail how to achieve this step by step using VanillaJS and Bun.

At Node Conference 2023, Jarred Sumner (creator of Bun) showed a demo of server components in Bun, so there is at least partial support in that ecosystem. The Bun repo provides bun-plugin-server-components as the official plugin for server components. And while I haven’t looked at it in-depth, Marz claims to be a “React Server Components Framework for Bun”.

Continuously evolving, Bun is currently optimized for MacOS and Linux, with ongoing efforts towards Windows compatibility. Tailored for resource-constrained environments like serverless functions, it emerges as an ideal solution. The Bun team is committed to achieving comprehensive Node.js compatibility and seamless integration with prevalent frameworks. For those intrigued by Bun's potential and want to give it a try, more information is available on its website at https://bun.sh/.

Let’s say you are interested in learning more about Bun and probably give it a try. Bun has a website, where you can learn more about Bun and its features (including all the benchmark data captured in this issue), and here is the link.

Looks like it, it seems the 2% are mostly odd platform specific issues that the authors' did not deem very important (my assumption for the release happening anyway). AFAIK this[1] PR tries to fix them.

[1]: https://github.com/oven-sh/bun/pull/9729

Bun has a solution for it. First of all, it already has a list of trusted dependencies. For them, Bun will execute all necessary scripts by default. Otherwise, you can add it to trustedDependecies in your package.json file. In Bun community usage of trustedDependencies is a hot topic. There are several suggestions on how to improve it.

Install Node.js (or Bun, or Deno, or whatever JS runtime you prefer) if it's not there

I think maybe I was unclear. I'm talking about writing libraries that abstract across these differences and provide a single API, as sibling describes. I already know it's possible. I made a simple filesystem abstraction here[0] and a very simple HTTP library that uses it here[1]. They both work in Node/Deno and the browser. Unfortunately I ran into issues with Bun's slice implementation[2]. But I suspect there's a much better way of detecting and using the different backends.

[0]: https://github.com/waygate-io/fs-js

[1]: https://github.com/waygate-io/http-js

[2]: https://github.com/oven-sh/bun/issues/7057