pulldown-cmark VS Rustler

As a platform that allows expressiveness, we want our users to be bold enough to ask and answer questions with either plain text or some markdowns. Compiling markdown to HTML in Rust can be done via the pulldown-cmark crate. We used it in this utility function:

I also really like this paradigm. It’s just that in old crusty null-terminated C style this is really awkward because the input data must be copied or modified. But it’s not an issue when using slices (length and pointer). Unfortunately most of the C standard library and many operating system APIs expect that.

I’ve seen this referred to as a pull parser in a Rust library? (https://github.com/raphlinus/pulldown-cmark)

Really glad to see this! Really want an easy way to render graphs in Rust without resorting to the graphiz binary.

What is the current status? Not seeing it listed anywhere, like if there are features that are not supported or if it uses certain layout algorithms but others are desired.

Would you be willing to make a `[lib]` available? I see you have a `lib.rs` but it'd be great if using it didn't require pulling in `[[bin]]` dependencies (you can mark them as optional and mark `required-features` on your bin like pulldown-cmark does [0] or split it into a separate crate in a workspace). It'd also be good to find an available name for the lib and get it published (looks like someone might be squatting on `layout`).

[0] https://github.com/raphlinus/pulldown-cmark/blob/master/Carg...

Author here. I actually was not aware of cmark.ex - thanks for pointing it out.

In this case the code reuse was more important than pure native speed. We already had a Rust library that used pulldown-cmark [1] with some custom tweaks that we wanted to duplicate. Maybe this behavior could have been copied using cmark.ex too (we thought about doing this in pure Elixir, as mentioned in the post), but given how straightforward Rustler made integrating our existing code, this seems like the better choice.

[1] https://github.com/raphlinus/pulldown-cmark

It turned out that making the most popular Elixir Markdown processor, Earmark (originally written by Dave Thomas) and pulldown-cmark, a Rust Markdown processor, produce the same output was going to be difficult. We also required some customization that was not available in both libraries.

The crate that's closest to production quality code is pulldown-cmark, but I don't hold it up as an example of well-written code, because it's not particularly easy to understand and there's a lot of very low level code to consume the CommonMark syntax - that helps with code bloat and compile time, but not clarity.

The Markdown processor is pulldown-cmark, which supports these extensions:

For performance intensive tasks, you could rely on Rust NIFs, there is this great project: https://github.com/rusterlium/rustler

My last project with Elixir was using Elixir merely as an orchestrator of static binaries (developed in golang) which were talking in JSON via stdin/stdout.

From the moment we discovered Tauri, we really felt like this was the perfect fit. The API is really solid, the configuration files are minimal and easy to understand, and the usage of Rust makes it way easier to add new functionalities and think about interesting ways of interoperating with Elixir via the Rustler library.

Elixir/Rust is the new Python/C++, and Rustler makes the communicating between the 2 languages super easy: https://github.com/rusterlium/rustler

Rustler is so awesome for this. Write Elixir NIFs in Rust? Yes, please!

But really, I would suggest thinking about what you want to build before "how" or "with which tool" - one of the signs of a person becoming a good engineer is having an array of tools at their disposal and being able to choose a correct tool for the correct task. Rust also excels in integrating with other languages - with JS via WebAssembly (a bit of self-promotion, for example), with Elixir via Rustler, with Python via PyO3 and PyOxidizer, etc. So you absolutely can start writing a frontend app with JS, or a distributed system with Elixir, or a data processing/ML app with Python and use Rust to speed up critical parts of those. Or, in reverse, you can start with Rust & add new capabilities to whatever you're building, that being a frontend, a resilient chat interface, or an ML model.

I created a elixir (wrapper) library to generate password hashes. Other Elixir libraries use a C NIF to generate password hashes. This libary uses a Rust NIF (using Rustler) and the Rust libraries the generate the different hashes. Additionally this library uses RustlerPrecompiled so you don't need to have a Rust compiler installed to use this library. It supports argon2, scrypt, brypt and pbkdf2.

> I guess, why not use Rust entirely instead of as a FFI into Elixir or other backend language?

Because Rust brings none of the benefits of the BEAM ecosystem to the table.

I was an early Elixir adopter, not working currently as an Elixir developer, but I have deployed one of the largest Elixir applications for a private company in my country.

I know it has limits, but the language itself is only a small part of the whole.

Take ML, Jose Valim and Sean Moriarity have studied the problem, made a plan to tackle it and started solving it piece by piece [1] in a tightly integrated manner, it feels natural, as if Elixir always had those capabilities in a way that no other language does and to put the icing on the cake the community released Livebook [2] to interactively explore code and use the new tools in the simplest way possible, something that Python notebooks only dream of being capable of, after a decade of progress

That's not to say that Elixir is superior as a language, but that the ecosystem is flourishing and the community is able to extract the 100% of the benefits from the tools and create new marvellously crafted ones, that push the limits forward every time, in such a simple manner, that it looks like magic.

And going back to Rust, you can write Rust if you need speed or for whatever reason you feel it's the right tool for the job, it's totally integrated [3][4], again in a way that many other languages can only dream of, and it's in fact the reason I've learned Rust in the first place.

The opposite is not true, if you write Rust, you write Rust, and that's it. You can't take advantage of the many features the BEAM offers, OTP, hot code reloading, full inspection of running systems, distribution, scalability, fault tolerance, soft real time etc. etc. etc.

But of course if you don't see any advantage in them, it means you probably don't need them (one other option is that you still don't know you want them :] ). In that case Rust is as good as any other language, but for a backend, even though I gently despise it, Java (or Kotlin) might be a better option.

[1] https://github.com/elixir-nx/nx https://github.com/elixir-nx/axon

[2] https://livebook.dev/

[3] https://github.com/rusterlium/rustler

[4] https://dashbit.co/blog/rustler-precompiled

unfortunately as of the time of this writing, rustler does not support generic type intefaces so I guess this is impossible?

One thing that drew me to Rust was actually Elixir/Erlang calling out to it for certain specialized needs. Within Elixir/Erlang you get best of breed concurrency but exiting the BEAM to run other code is unsafe. Calling out to Rust, however, comes with great safety guarantees.

Managing concurrency outside of Rust and then calling Rust for the more focused and specialized work is a good combination IMO.

https://github.com/rusterlium/rustler