one-more-re-nightmare VS SICL

Depends on your regex engine, and your non-regex solution. My engine (shameless self-plug https://github.com/telekons/one-more-re-nightmare) rivals hand-written automata, having to load each character more-or-less* only once, and throws in vectorisation for simple search loops too. I would not want to write or maintain the generated code.

one-more-re-nightmare used to let you write Σ, but I then tried to search Greek stuff with it and it went wrong. So now there's...$ for all characters (since that's not used for end-of-line assertions).

One trick is that Common Lisp can generate and compile code at runtime, whereas static languages typically do not have a compiler available at runtime. This lets you make your own lazy person's JIT/staged compiler, which is useful if some part of the problem is not known at compile-time. Such an approach has been used at least for array munging, type munging and regular expression munging.

> One of my favorite examples is the regex library cl-ppcre. Thanks to the nature of Lisp, the recognizer for each regex you create can be compiled to native code on compiler implementations of CL.

That is not true - cl-ppcre generates a chain of closures. Experimental performance is in the same ballpark as typical "bytecode" interpreting regex implementations.

(Disclosure: I wrote another regex library at <https://github.com/telekons/one-more-re-nightmare>, which does do native code compilation.)

Someone else didn't tell me that before, so it can't be true. But I don't publish papers on toys, nor do I think toy projects are awfully fast. Though the x86-64 backend I wrote was in someone else's repository and thus was several PRs :(

Regular expressions

Also try this https://github.com/telekons/one-more-re-nightmare

A Common Lisp system has the compiler around at runtime, so if you can figure out how to profitably stage/specialise a computation, then you can roll your own cheap JIT of sorts. This can be useful for array munging and regular expressions at the least. You can do this in C, of course but you would need to use another compiler as a library (e.g. LLVM, TCC, libgccjit) or write your own (e.g. PCRE2's sljit).

Agree to disagree - I don't have the energy to remember operator precedence. One file from the regular expression compiler has most of the rewrite rules I read from the papers, except in S-expression syntax. There were a few bugs due to misreading precedence. Also c.f. Gerald Sussman talking about physics notation being a pain in the butt.

It's all part of the library. Everything about regular expression types is in this file.

This is a very approachable paper from 1990 on one way to do it with a C kernel bootstrapping to Common Lisp: https://www.softwarepreservation.org/projects/LISP/kcl/paper... Kyoto Common Lisp (KCL) is the ancestor of today's Embeddable Common Lisp (ECL).

SICL is probably the best modern version of CL written in CL from a design standpoint, even if it's not taking over SBCL's role anytime soon: https://github.com/robert-strandh/SICL It uses some fancy bootstrapping to have the whole language available early, e.g. their definition of class 'symbol is:

    (defclass symbol (t)

That's pretty much the objective of SICL, which is "intentionally divided into many implementation-independent modules that are written in a totally or near-totally portable way, so as to allow other implementations to incorporate these modules from SICL, rather than having to maintain their own, perhaps implementation-specific versions".

   https://github.com/robert-strandh/SICL

Gladly!

https://github.com/robert-strandh/SICL (which I wrote a decent chunk of the compiler backend of.)

I applaud your curiosity and initiative to explore. Are you aware of https://github.com/robert-strandh/SICL?

I mean this Klein and this SICL. Self and Common Lisp are memory-safe, though the implementations need capabilities to manipulate memory; SICL encapsulates them using first-class global environments.

There is significant overlap with SICL and its associated pieces which supply many of the other parts needed to make a Common Lisp. Some of these are Cluster which provides a portable and extensible assembler, Eclector which supplies a portable and extensible reader, Concrete-Syntax-Tree that supports source code tracking during compilation, ctype that implements the Common Lisp type system, and Clostrum that provides first-class environments for e.g. run-time, evaluation, and compilation. The SICL project has as one of its goals the creation of portable infrastructure for implementing Common Lisp, and these pieces are novel building blocks that were created as part of the project.

Not really; you can do it with primitive operations e.g. here is the list in the Cleavir compiler and a paper on "magic" in Jikes RVM. SBCL also has a "virtual op"/vop language for code generation, and vops are written to manipulate objects with assembly snippets.

Both work. I basically never have to touch C or even FFI (cl+ssl being the main use of FFI for me), unless I am poking at SBCL guts in my spare time, and that isn't necessary either. I am sure many Haskell hackers are happy with their IO monad too.

It's used in operating systems, compilers and CLIs.

The Cleavir Common Lisp compiler uses three-address instructions in a control-flow graph, though it is intended more for production use than educational use.