Fast Parse VS IParse

Perhaps use Li Haoyi's fastparse instead? https://github.com/com-lihaoyi/fastparse

You might want to adapt Li Haoyi’s XML parser for fastparse.

Mostly a mess of pattern matching. I really need to make some generic tree utilities. Haven't been able to find a decent parser combinator that works in Scala 3 (I usually use fastparse which depends heavily on Scala 2 macros, and scala-parser-combinators works in Scala 3, but I've had a lot of trouble getting it to not be too greedy), so I used the state monad from cats to parse at the bottom of the file, which I think turned out fairly nice.

Agreed! I would say that parser combinators are the sweet spot and the right choice in most cases.

Scala has them as well, e.g.: https://com-lihaoyi.github.io/fastparse/

And the good thing is, you don't have to learn a completely new language/syntax, you can use the host language's syntax and you have full IDE support as well.

Nice library with many features. But I do not always understand the focus on memory usage. I guess that the reason behind this is that less memory allocations, have a positive effect on execution times. In a parser, where you often have to compare identifiers, it is a good idea to put all strings for identifiers into a unique pointer with the help of a hash table.

In my interpreting parser [1] I use a hexa hash tree [2] for storing identifiers. It is not very memory efficient, but very fast. It turns every string (from the input buffer) into a unique pointer for that string pointing to a copy of the string. In this way comparing string (identifiers) is equivalent to comparing pointers.

The idea of the hexa hash tree is that is a tree where each node has sixteen child nodes. Which node is selected is based on a step wise evaluated hash function that first takes the lower four bytes of the string, and after reaching the end of the string, the higher four bytes of the string. The nodes often taken up more memory space than the strings themselves.

[1] https://github.com/FransFaase/IParse/

[2] https://github.com/FransFaase/IParse/blob/master/software/Id...

I wrote a tool that can process a number of MarkDown files with fragments of C code and put all those fragments in the right order to produce a file that can be compiled. It is grammar based and works with manipulating Abstract Syntax Trees, so I guess, it could be adapted for different programming languages. See: https://github.com/FransFaase/IParse#markdownc

Interesting. I have not looked into the code, but I wonder whether it is a compiler, or just an interpreter, e.g. it converts the grammar into some internal representation that is executed by an interpreter or virtual machine. I started worked on an interpreting parser in C many years ago. And later also made Java, C++ and JavaScript version of it. For the JavaScript implementation, see: https://fransfaase.github.io/ParserWorkshop/Online_inter_par... For the C++ version, see: https://github.com/FransFaase/IParse

I implemented an unparse function in IParse, which is not a parser generator, but a parser that interprets a grammar. See for example https://github.com/FransFaase/IParse/blob/master/software/c_... where symbols starting with a back slash are a kind of white space terminals during the unparse. For example, \inc stands for incrementing the indentation where \dec decrements it. The \s is used to indicate that at given location a space should be included.