Lark VS cupscript

Lark supports, and recommends, writing and storing the grammar in a .lark file. We have syntax highlighting support in all major IDEs, and even in github itself. For example, here is Lark's built-in grammar for Python: https://github.com/lark-parser/lark/blob/master/lark/grammar...

You can also test grammars "live" in our online IDE: https://www.lark-parser.org/ide/

The rationale is that it's more terse and has less visual clutter than a DSL over Python, which makes it easier to read and write.

First, let me introduce myself. My name is Erez. You may know some of the Python libraries I wrote in the past: Lark, Preql and Data-diff.

It's not super maintained but you might enjoy building something with ppci, Pure Python Compiler Infrastructure. It has some front-ends and some back-ends. There's also PeachPy for an assembler. People like using Lark for parsing, I hear.

lark, a parsing library where I am somewhat involved has a really nice solution to this: Rules starting with _ are inlined in a post processing step.

Lark is a good library to assist with this.

If I may ask, did you consider Lark, and if so, why wasn't it fit for your purposes?

A grammar provides the high level constructs you need to define the "shape" of your data, and it largely takes care of the rest. Grammar libraries exist in other language (eg. lark or Parsimonius in Python) and they weren't created just to make XML parsing easier.

I made a solid attempt at an Earley parser framework of my own, but apparently to get the most reliable performance from Earley parsing you need to implement Joop Leo's improvement for right-recursive grammars, which nobody has been able to adequately explain to me. I've read Kegler's open letter to Vaillant, I've tried to read other implementations, I've even tried to beat my head against the original academic paper, but I don't have the background knowledge to make sense of it all.

Oh nice I followed that tutorial and created it from scratch few years back also shared here, its a pretty good tutorial but at that time I was kind of blindly following and didn't understand much but later updated with import statements, implicit variable declarations, and something like python's if __name__ == "__main__" and a File Handler objectit was pretty fun ngl now I am studying programming languages properly and trying to make one,you can check it here https://github.com/Fus3n/cupscripteverything I did in a way it may not be useful but you can look into the code and extend from it, it probably looks exactly the same (btw mine has some bugs)

Lexer or also called a tokenizer is basically the first step of all of it. A lexer class takes in the text/user input (also filename for context to throw errors in my case) and goes through every single character and checks if any of it matches with the character I pass in if statements, if it does then if it is a '+' for example it will append a new token to a list (mine is called tokens) using the Token class you can check that out too a token class takes a Token type and a value so the type is PLUS here and the value is '+' all my types are stored in consts.py and then the tokens get passed to the parser