grammars-v4 VS coherence

Perhaps the one from ANTLR's collection [0] is a good start (there are also others ANTLR VB6 grammars documented elsewhere). It does require knowing ANTLR, but that should be less effort for someone already familiar with language implementation, particularly, the visitor pattern (my favorite reference [1]).

[0] https://github.com/antlr/grammars-v4/tree/master/vb6

[1] https://craftinginterpreters.com/representing-code.html

Where is the SQLite test suite, please? I'd be very interested.

There are already SQL grammars, check https://github.com/antlr/grammars-v4 specifically in here I think https://github.com/antlr/grammars-v4/tree/master/sql I contributed to one of them, and I wrote my own for some personal work. Be warned, it's very involved, very complex and MSSQL is rather ill-defined.

Names bracket identifiers) in SQL are bloody awful. Sometimes square brackets are even compulsory, and why you can usually replace [...] with the SQL standard "..." , not always! Trust me, it gets worse.

I don't find antlr grammars to be brittle, and while they can lose in performance (by how much I don't know, perhaps quite considerably) they are very easy to maintain and I am very fortunate to have antlr to work with.

This grammar "library" was cited as an example of what the format could look like:.

https://github.com/antlr/grammars-v4

There is everything from assembly and C++ to glsl and scripting languages, arithmetic, games, and other weird formats.

> Which brings me to the other approach: steering the LLM's output __as it is generating tokens__

A relevant PR:

https://github.com/ggerganov/llama.cpp/pull/1773

The plan is to support arbitrary grammar files to constrain tokens as they are generated, like the ones here:

https://github.com/antlr/grammars-v4

Have a look at jooq - I know this has been used to rewrite SQL from one dialect to another, so it MUST be capable of collating code activity metrics. Look here. Otherwise, you might want to look into writing your own parser. ANTLR has a T-SQL dialect parser script here.

Find a Perl grammar file for ANTLR, like https://github.com/antlr/grammars-v4/tree/master/perl Save the grammar file as Perl.g4 in your project. Now, you can create the Kotlin program: import org.antlr.v4.runtime.* import org.antlr.v4.runtime.tree.ParseTree import java.io.File

Antlr is another option. You could generate a parser using the JSON antlr grammar.

There are a few different tool-sets for producing Java byte code. I'm not sure which one to suggest, because back when I last needed one (end of '96), there were none, so I wrote my own. But I assume that most people use ASM or something similar.

This is one that I like a lot. Years ago (1997 timeframe) I had implemented it in a Java compiler, and a few years later in a Java library (https://github.com/oracle/coherence/blob/4e6e343e1ffd9bbfea3...) that would create an exception on the assertion failure and parse its stack trace to find the source code file name, and read it to find the text of the assertion that failed, etc. so it could build the error message ...

In Ecstasy, we built the support directly into the compiler again:

```

For the debugger (but not required by the runtime), there is an optional table that points to the ranges of ops at which names and types are bound to registers

FWIW - here's an AST for Java that directly emits Java byte code: https://github.com/oracle/coherence/tree/master/prj/coherence-core/src/main/java/com/tangosol/dev/compiler/java

It has been "experimented with" many times. Here's an example from TDE, a component-based development environment from Tangosol (now part of Oracle).

If it helps, here's a Java recursive descent parser that I wrote years ago.

If you want to see an example, here's a Context interface from a multi-language compiler framework (compiling multiple different languages to Java byte-code) that I wrote years ago.

I built an entire Java compiler in four months, from scratch, by myself, over twenty years ago. (Now owned by Oracle; still used today. Thank you, Larry.) But starting from a well written spec for a simple language like Java is orders of magnitude easier than developing a new language, runtime model, and tool-chain from scratch.

As mentioned, the various parsing methods each contribute back an AST node, so on the way down the recursion, they are parsing, and on the way back up from the recursion, they are building the tree. Here's a fairly simple recursive descent Java compiler written in Java that I wrote a few years back, in case you are looking for an example.

If you're curious how some of this can be implemented in a Java compiler, I wrote one years ago. For example, checking that the left side is an l-value: