bitvec VS nom

I was also under the false impression that bitwise encoding was slow. When I first implemented bitcode with bitvec I got performance 20x worse than bincode. After writing my own implementation I was able to get much better performance.

interesting; i'll have to compare this to my rust counterpart. your numbers indicate some clever implementations i'd love to read

Not to mention having a lackluster std which causes you to use nonstardard not so well documented crates and a 40K LoC library to do "bit-twiddling" (the lib, https://github.com/bitvecto-rs/bitvec the blog that says "twiddle bits" https://blog.adamchalmers.com/making-a-dns-client/ and for crying out loud the blogger also used the language the author mentioned and I quote "ergonomics AND speed AND correctness")

https://github.com/bitvecto-rs/bitvec/issues/135 is a very funny read about how to perform inttoptr with provenance retention

Technically #135 gives me license to yank affected crates, but since the only exploit is "Miri crashes exactly one test out of the suite" it's not really worth it to be a stickler. Call it a truce

My friend Nika wrote a macro that packs a sequence of 1, 0, … tokens into a correctly structured bit-buffer, adaptable over any register type or bit-ordering, at compile time. It's now basically this whole file

if you are interested in a user manual, you can use mdbook as well. for an example, my bitvec project uses mdbook (book.toml) and a github action (.github/workflows/gh-pages.yml) to compile the guide and host it as a github pages website. it's slightly more complicated, and i'd like docs.rs to follow hexdoc.pm's example of hosting both api docs and prose, but until then this is a pretty reasonable solution.

the first one

i would browse the standard library, tower, nom, or my own bitvec to see layout and trait/record separation. in particular, std::io and std::net may be of use: io::Read and io::Write are pervasive examples of implementing unixy file-descriptor-like behavior in the type system

Just in case you are not familiar with nom, it is a parser combinator written in Rust. The most basic thing you can do with it is import one of its parsing functions, give it some byte or string input and then get a Result as output with the parsed value and the rest of the input or an error if the parser failed. tag for example is used to recognize literal character/byte sequences.

I may be the only one not familiar, but nom refers to https://github.com/rust-bakery/nom which looks like a pretty handy way to parse binary data in Rust.

Lots of people use nom for their parsing needs, but that's not the only game in town and there other options.

As much as I love nom as well as other parser combinator libraries, regex-based parsers, BNF/EBNF-based parsers, etc. I always end up going back to plain old text-based char-by-char scanners.

I am using nom / nom_locate to build the parser side because I've done a handful of other projects with it, and I plan to use tower-lsp to hook up the language server side.

Look into a parsing library such as https://github.com/rust-bakery/nom

If we were in /r/learnrust I'd have recommended the nom crate for this.

Thanks to the flexibility of [nom](https://github.com/rust-bakery/nom), it is very easy to write your own parser in rust, read [this article](https://github.com/metaworm/luac-parser-rs/wiki/Write-custom-luac-parser) to learn how to write a luac parser

I've been working on an assembler and right now it uses nom. While nom isn't great for error messages, good error messages will be important for this particular assembler (current code), so I've been attempting to use the methods described by Eyal Kalderon in Error recovery with parser combinators (using nom).