nom VS chumsky

And on a related note, this project was for educational purposes, but if the author wants to do more parsing in Rust, there's the excellent `nom` crate [1], which provides a JSON parser as an example [2].

It uses a very similar paradigm to what the author used in the article, and provides a lot of helper utilities. I used it to parse a (very, very small) subset of Markdown recently [3] and enjoyed the experience.

[1] https://github.com/rust-bakery/nom

[2] https://github.com/rust-bakery/nom/blob/main/examples/json.r...

[3] https://git.sr.ht/~bsprague/logseq-to-linkwarden

To see these concepts in action, check out my repository Fun with Nom. It’s a project where I explored the Nom crate during a deep dive early in 2024.

Part of that inspiration was learning about parser combinators, particularly Rust's nom library. Of course, parser combinators - particularly in Rust - presented a bit of a learning curve. But this incredible post by Bodil Stokke really helped me understand how they worked, and made me feel empowered. In fact, for small DSLs, I often use parser combinators - for instance, with ts-parsec in TypeScript and parsy in Python. If you want to dip your toes in and have a head for functional programming DSLs, this is a great direction to go in.

Unlike the previous parser, which was based on the nom parser-combinator library, the new parser is an optimised recursive descent parser with a separate lexing step. This change allows for more efficient parsing by separating the tokenisation of the input from the parsing logic itself, streamlining the parsing process.

Just in case you are not familiar with nom, it is a parser combinator written in Rust. The most basic thing you can do with it is import one of its parsing functions, give it some byte or string input and then get a Result as output with the parsed value and the rest of the input or an error if the parser failed. tag for example is used to recognize literal character/byte sequences.

I may be the only one not familiar, but nom refers to https://github.com/rust-bakery/nom which looks like a pretty handy way to parse binary data in Rust.

Lots of people use nom for their parsing needs, but that's not the only game in town and there other options.

I attempted to use this but was disheartened but the fact that it doesn't statically type node names. Tree Sitter doesn't either but it has much more of an excuse given that it targets C.

https://github.com/lezer-parser/lezer/issues/8

The dev seems mildly hostile to outside involvement too, so I moved on. These days I use Chumsky which is Rust rather than Typescript, but also way more awesome, if you can deal with the often incomprehensible compilation errors at least!

https://github.com/zesterer/chumsky

there’s also chumsky: https://github.com/zesterer/chumsky

We are using technique called parser combinator. And we are using a library chumsky to write parser combinators.

rust-langdev has a lot of libraries for building compilers in Rust. Perhaps you could use these to make your implementation easier, and revisit it later if you want to build things from scratch. I'd suggest logos for lexing, LALRPOP / chumsky for parsing, and rust-gc for garbage collection.

The examples that come with chumsky and the chumsky tutorial and guide all define their parsers using closures.

Parser: https://crates.io/crates/chumsky

chumsky's zero-copy rewrite has reached its first alpha release, and I have migrated my pep-508 parser to it, as suggested in my last announcement.

On my side, nom is still advancing well and a new major version is in preparation, with some interesting work a new GAT based design inspired from the awesome work on chumsky, that promises to bring great performance with complex error types. 2023 will be fun for parser libraries!

I am using chumsky because I like the API, but it doesn't support zero copy at the moment. Although efficiency is good to have, it is not my primary good. This will probably get supported once chumsky implements support for it (see upstream issue).