go-mysql-server VS Checker Framework

We implemented a query optimizer with a flexible intermediate representation in pure Go:

https://github.com/dolthub/go-mysql-server

Getting the IR correct so that it's both easy to use and flexible enough to be useful is a really interesting design challenge. Our primary abstraction in the query plan is called a Node, and is way more general than the IR type described in the article from OP. This has probably hurt us: we only recently separated the responsibility to fetch rows into its own part of the runtime, out of the IR -- originally row fetching was coupled to the Node type directly.

This is also the query engine that Dolt uses:

https://github.com/dolthub/dolt

But it has a plug-in architecture, so you can use the engine on any data source that implements a handful of Go interface.

Might be interested in https://github.com/dolthub/go-mysql-server, which also does this

https://github.com/dolthub/go-mysql-server

One item under "Scope of this project":

Provide a runnable server speaking the MySQL wire protocol, connected to data sources of your choice.

I just wanted to say thanks for https://github.com/dolthub/go-mysql-server

This is incredibly useful for anyone who wants to build their own DB or wrap another datasource so it's queryable via MySQL protocol.

a very cool project they also maintain is a MySQL server framework for arbitrary backends (in Go): https://github.com/dolthub/go-mysql-server

You can define a "virtual" table (schema, how to retrieve rows/columns) and then a MySQL client can connect and execute arbitrary queries on your table (which could just be an API or other source)

Thanks for this write up! I've been really interested in postgres compatibility in the context of a tool I maintain (https://github.com/mergestat/mergestat) that uses SQLite. I've been looking for a way to expose the SQLite capabilities over a more commonly used wire-protocol like postgres (or mysql) so that existing BI and visualization tools can access the data.

This project is an interesting one: https://github.com/dolthub/go-mysql-server that provides a MySQL interface (wire and SQL) to arbitrary "backends" implemented in go.

It's really interesting how compatibility with existing protocols has become an important feature of new databases - there's so much existing tooling that already speaks postgres (or mysql), being able to leverage that is a huge advantage IMO

It is interesting! I experimented with creating a bad borrow checker for Java using annotations from

https://checkerframework.org/

It supports some level of substructural types using must-call annotations,

https://checkerframework.org/manual/#resource-leak-checker

Lombok is not a compiler extension. Compiler extensions, aka annotation processors, are offered only specific capabilities that ensure that they preserve the Java language specification. Particularly, code that compiles successfully with an extension also compiles without it (perhaps requiring other classes to be available) and it compiles down to the same bytecode. Annotation processors are used to implement pluggable type systems (e.g. https://checkerframework.org) or to generate other classes (e.g. https://immutables.github.io/).

Unlike compiler extensions, Lombok compiles source files that do not conform to the Java language specification. Lombok is an alternative Java Platform language, like Clojure or Kotlin or Scala, except that it's a superset of the Java language. However, rather than forking `javac` source code and modifying it to compile Lombok source files, the Lombok compiler modifies `javac`'s operation by hacking into its internals and modifying them as it runs to compile Lombok sources rather than Java sources.

Having alternative Java Platform languages is perfectly fine. The problem with Lombok is that it doesn't present itself as such but as a library or a compiler extension even though it violates the Java language specification in ways that compiler extensions are forbidden from doing.

And then I found (thanks Oracle), https://checkerframework.org/ zomg, this thing is awesome. Pluggable Type Systems!

Java should adopt something like the Checker Framework Nullness Checker in its first-party tooling.

https://github.com/typetools/checker-framework

And modern (real, non-android) Java via project amber and so on has gone more and more quasi functional with its immutability and sealed and record types for effective sum types, as well as its pretty cool type-use annotation extensible static type checks.

Checkerframework - a really academic take, and as one might expect from such a thing, backed by tons of papers and analysed to perfection. Specifically, this is the only framework I'm aware of that realizes nullity is a little more complicated than just a boolean yes-or-no; just like generics actually have 4 flavours for any given type: List, List, List, and List are all 4 important and unique, and nullity is no different. Specifically, it can occur that you want to write a method that ought to accept both lists of nullable strings as well as list of nonnull strings, and needs to 'convey' this nullity again on its output. You can either attempt to lift along the existing generics system in java which I think is your intent, but it's not actually all that easy to do this. After all, T extends @Nullable Number super @NonNull Number, or whatnot, isn't legal java. So you.. really just can't do that. Checker Framework solves this problem by introducing the @PolyNull annotation, which still isn't perfect but covers almost all real world use cases you can think of. I'm missing any acknowledgement in your documentation. An oversight, or, something you hadn't thought of yet? You're in good company: Both eclipse and intellij's engineers, when I asked them about it, just hadn't realized it was a thing. Point is: If you think the primary problem with e.g. eclipse's and intellij's take is that they lack academic rigour - checkerframework has you beat.

NullAway or the Checker Framework should greatly help eliminate the issue. Also, when Java gets value types you should be able to define your own non nullable value types and use them safely.

There are a lot of existing libraries for type checking modules. For example the Checker Framework created by University of Washington. This framework includes a NonNull module, as well as regular expression module and a mutex lock module. See this for more information.