bstr VS wasi-libc

Libraries for a lot of this stuff exist (albeit in many cases not very mature yet):

- https://github.com/pop-os/cosmic-text does text layout (which Taffy explicitly considers out of scope)

- https://github.com/AccessKit/accesskit does accessibility

- https://github.com/servo/rust-cssparser does value-agnostic CSS parsing (it will parse the general syntax but leaves value parsing up to the user, meaning you can easily add support for whatever properties you what). Libraries like https://github.com/parcel-bundler/lightningcss implement parsing for the standard css properties.

- There are crates like https://github.com/BurntSushi/bstr and https://docs.rs/wtf8/latest/wtf8/ for working with non-unicode text

We are planning to add a C API to Taffy, but tbh I feel like C is not very good for this kind of modularised approach. You really want to be able to expose complex APIs with enforced type safety and this isn't possible with C.

As the author of bstr and also the regex implementation that bstr uses to implement word breaking, it is linear time.

NSFL: https://github.com/BurntSushi/bstr/blob/86947727666d7b21c97e...

OsStr uses WTF-8 on Windows, and just represents the raw underlying bytes on Unix.

Byte strings can be WTF-8. They can be anything. The problem is that there is no real way to (easily) get the underlying WTF-8 bytes of an OsStr on Windows. So there's no free conversion to and from byte strings.

I wrote more about this in the bstr docs (and don't miss the link to os_str_bytes): https://docs.rs/bstr/latest/bstr/#file-paths-and-os-strings

I'd be happy to answer more questions if you have them. :-) https://github.com/BurntSushi/bstr/discussions

Check out bstr, which does this exact thing for its BString and BStr types.

Currently slated for next Monday: https://github.com/BurntSushi/bstr/issues/40

This is just an FYI. I don't mean to say much to your overall point, although, as someone else who has spent a lot of time doing Unicode-y things, I do tend to agree with you. I had a very similar discussion a bit ago.[1]

Putting that aside, at least with respect to grapheme segmentation, it might be a little simpler than you think. But maybe only a little. The unicode-segmentation crate also does word segmentation, which is quite a bit more complicated than grapheme segmentation. For example, you can write a regex to parse graphemes without too much fuss[2]. (Compare that with the word segmentation regex, much to my chagrin.[3]) Once you build the regex, actually using it is basically as simple as running the regex.[4]

Sadly, not all regex engines will be able to parse that regex due to its use of somewhat obscure Unicode properties. But the Rust regex crate can. :-)

And of course, this somewhat shifts code size to heap size. So there's that too. But bottom line is, if you have a nice regex engine available to you, you can whip up a grapheme segmenter pretty quickly. And some regex engines even have grapheme segmentation built in via \X.

[1]: https://github.com/BurntSushi/aho-corasick/issues/72

[2]: https://github.com/BurntSushi/bstr/blob/e38e7a7ca986f9499b30...

[3]: https://github.com/BurntSushi/bstr/blob/e38e7a7ca986f9499b30...

[4]: https://github.com/BurntSushi/bstr/blob/e38e7a7ca986f9499b30...

This is a great idea. I realize the find implementation is not ideal and have considered bringing in an optional dependency to improve performance. I remembered bstr using two-way search, so I was wondering if depending on the full crate for searching would be worthwhile, but I see that changed. Thanks for the tip!

Fun little nit-pick that does not detract from your overall point: you can actually count graphemes with a regex and that's exactly what bstr does. :-)

Now there are projects like WASI that allows for interfacing with system resources for WASM code this allows for devs to target WASM runtime for their apps sliding the apps to run locally on any OS without any porting required. This could be a game changer in the future like Docker and containers was in the past decade.

So it doesn’t seem like there has been progress on a pseudo-random number generator function for typst, but there are multiple other ways to solve this: 1. Just don’t. Typst has this functional philosophy, there one input always produces the same output. (not an answer to your question tho) 2. Interface with a webassembly module which has a random number generator. So you could e.g. compile c to wasm and statically link a libc version. You would then just have to export the rand() function. (You could use any lang for this, which has a stdlib with a pseudo random number generator) 3. Implement your own. Random number generators are actually not that hard something like an LCG isn’t to complex. (Id provide an example but im on my phone rn)

Actually WASI[0] will be a better alternative, IIRC extism serialize and deserialize the data that you want to pass every time, adding a lot of overhead.

[0] https://wasi.dev

Actually, it was in wasi-libc: https://github.com/WebAssembly/wasi-libc/blob/main/libc-bott...

Proper isolation in C# is only now becoming a thing, with .Net support for WASI, which is essentially a WebAssembly sandbox which can be given extremely granular privileges (such as access to spefic file system directories, or an effective virtual file system). As an upside, the idea is that it should be possible to write the WASI packages in more or less anything.

Wasm Labs dev here :)

In mod_wasm, there are some differences with a pure CGI implementation. When Apache boots, it loads the configuration and initializes the WasmVM. When a new HTTP request arrives, the VM is ready so you don't need to initialize a different process to manage it.

You still need to process the request and pass the data to the Wasm module. This step is done via STDIN through the WebAssembly System Interface (WASI) implementation [0]. The same happens in the opposite direction, as the module returns the data via STDOUT.

So, the CGI pattern is still there, but it doesn't require new processes and all the code runs in a sandbox.

However this is not the only way you can run a Wasm module. In this specific case, we use CGI via WASI. In other cases, you may compile a module to fulfill a specific API, like ProxyWasm [1] to create HTTP filters for proxies like Envoy.

- [0] https://wasi.dev/

- [1] https://github.com/proxy-wasm/spec

Never done that, but I think you need this: https://wasi.dev/

The Python builds from the WebAssembly language runtimes [0] project target the WebAssembly System Interfaces (WASI) [1]. It allows the Python interpreter to interact with resources like the filesystem.

Many server-side Wasm runtimes supports WASI out of the box. For the browser, you need to provide a polyfill to emulate these resources like the one provided by the WASI team [2].

Regarding SQLite, these builds include libsqlite so you should be able to use it :)

- [0] https://github.com/vmware-labs/webassembly-language-runtimes

- [1] https://wasi.dev/

- [2] https://wasi.dev/polyfill/

Effective debugging results in effective programming; itk-wasm makes effective debugging of WebAssembly possible. In this tutorial, adapted from the itk-wasm documentation, we walk through how to debug a C++ data processing pipeline with the mature, native binary debugging tools that are comfortable for developers. This is a fully featured way to ensure the base correctness of a processing pipeline. Next, we will walk through an interactive debugging experience for WASI WebAssembly. With itk-wasm, we can debug the same source code in either context with an interactive debugger. We also have a convenvient way to pass data from our local filesystem into a WebAssembly (Wasm) processing pipeline.

Communication between the WASM module and the rest of the application needs to be done in very simple types (bytes, ints and floats). No complex types are supported yet. This is why most WASM compilers also provide some glue-code to map between complex types like strings or arrays. The Web Assembly System Interface (WAS) is an on-progress standard aimed to solve this last limitation; once it's mature it will allow easy interoperation with almost every environment. WASI is already available in some WSAM compilers and runtimes.