logos VS nom

This is mighty impressive! I've been trying to get some motivation for the mythical rewrite of the proc macro in Logos, and this might just do it for me :D. I'll have a proper look later today and see if any of your findings have something that can be generalized. Also really surprised to see aarch64 doing better than x86_64 since the latter is what I've been optimizing for!

Rust is a very nice langage for implementing compilers, and has a nice ecosystem for it (logos, rust-peg, lalrpop, astmaker -- this one is mine --, etc...).

rust-langdev has a lot of libraries for building compilers in Rust. Perhaps you could use these to make your implementation easier, and revisit it later if you want to build things from scratch. I'd suggest logos for lexing, LALRPOP / chumsky for parsing, and rust-gc for garbage collection.

Thanks! For compile times you might find the CLI version that Andrew Hickman contributed useful, it's undocumented still mostly I fear but shouldn't be hard to use, see original PR: https://github.com/maciejhirsz/logos/pull/248

For my lexer generation purposes, I tend to use https://github.com/maciejhirsz/logos, as it not only generates an easy to use lazy lexer, but the result is also exceptionally fast!

Hi, I'm using rowan to create a parser and want to print more useful error messages with position in the text/file. I'm using logos (https://crates.io/crates/logos) to generate the lexer. Is there a way to get the starting and ending positions of a SyntaxToken? If not I thought of adding my own wrapper struct around the SyntaxTokens.

Is there a way for a lexer created with the logos crate (https://crates.io/crates/logos) to get the starting and ending positions for the tokens?

I wouldn't recommend Logos at this point. This recent bug is quite nasty and seems easy to hit, and the maintainer is unresponsive. Last commit was half a year ago. At this point I consider Logos abandonware, though it would be great if its development continued, or if it were forked.

logos uses "ridiculously fast".

Just in case you are not familiar with nom, it is a parser combinator written in Rust. The most basic thing you can do with it is import one of its parsing functions, give it some byte or string input and then get a Result as output with the parsed value and the rest of the input or an error if the parser failed. tag for example is used to recognize literal character/byte sequences.

I may be the only one not familiar, but nom refers to https://github.com/rust-bakery/nom which looks like a pretty handy way to parse binary data in Rust.

Lots of people use nom for their parsing needs, but that's not the only game in town and there other options.

As much as I love nom as well as other parser combinator libraries, regex-based parsers, BNF/EBNF-based parsers, etc. I always end up going back to plain old text-based char-by-char scanners.

I am using nom / nom_locate to build the parser side because I've done a handful of other projects with it, and I plan to use tower-lsp to hook up the language server side.

Look into a parsing library such as https://github.com/rust-bakery/nom

If we were in /r/learnrust I'd have recommended the nom crate for this.

Thanks to the flexibility of [nom](https://github.com/rust-bakery/nom), it is very easy to write your own parser in rust, read [this article](https://github.com/metaworm/luac-parser-rs/wiki/Write-custom-luac-parser) to learn how to write a luac parser

I've been working on an assembler and right now it uses nom. While nom isn't great for error messages, good error messages will be important for this particular assembler (current code), so I've been attempting to use the methods described by Eyal Kalderon in Error recovery with parser combinators (using nom).