esy VS ocaml-multicore

As someone who wrote a fair amount of Rust and OCaml code, I have to agree with the author.

While working at Routine (YC W21), I was tasked with porting our core library to iOS to minimize code duplication. This was a lucky opportunity to write something resembling a compiler: it took in schemas described with our in-house data exchange library and generated C (for FFI) and Swift code (for the end-users, i.e., iOS developers).

Since Routine uses OCaml for everything (which was a big motivator for joining the company—I wanted to see how that would work out), I wrote it in OCaml. The end result is a 3-5k LOC project. It's by no means a full compiler, but it was lots of fun to write. The language got in the way incredibly rarely. On average, it made my life a lot easier.

We did encounter our fair share of issues, mostly due to the cross-compilation tooling (we initially used esy [1], flirted with Nix, and eventually switched to opam-cross-ios [2]), third-party libraries, and intricacies of FFI. Those do take their toll on sanity.

[1]: https://github.com/esy/esy/

What's the current status of Esy? https://github.com/esy/esy

Any plans to backport its design back to Opam?

It's great to hear that Opam is making progress! I just wished that it would be more deeply integrated with Dune. A package manager that doesn't build is not very useful to be honest. Currently the only way to not have to care about switches and be able to clearly specify dependencies is to use the esy package manager[1] (which had lock files a while ago).

[1]: https://github.com/esy/esy/

If you start a project today I would really try to use esy (https://esy.sh/)

I actually don’t use it myself but it seems to bring the modern programming language experience to OCaml

Here is link number 1 - Previous text "esy"

Alternatively, you can use esy.sh for a simpler setup/build process (it does not require running in a Cygwin shell).

Running esy does not work. Apparently, my environment does not know that it is there. Anyone know what is going on here? I have posted this in the discussion for esy@next here. I will get back to you all when I figure this out.

1. Domains are the unit of parallelism. A domain is essentially an OS thread with a bunch of extra runtime book-keeping data. You can use Domain.spawn (https://github.com/ocaml-multicore/ocaml-multicore/blob/5.00...) to spawn off a new domain which will run the supplied function and terminate when it finishes. This is heavyweight though, domains are expected to be long-running.

2. Domainslib is the library developed alongside multicore to aid users in exploiting parallelism. It supports nested parallelism and is pretty highly optimised (https://github.com/ocaml-multicore/domainslib/pull/29 for some graphs/numbers). The domainslib repo has some good examples: https://github.com/ocaml-multicore/domainslib/tree/master/te...

3. We've not tested against other forms of parallelism. There isn't anything stopping you exploiting SIMD in addition to parallelism from domains.

4. No, we've not compared performance by OS.

5. No plans for the multicore team to look at accelerator integration at the moment.

For Algebraic Effects and Multicore OCaml specifically, I have this intro saved and they've been publishing regular updates here's October's. They have a paper linked from their repo's README but I don't remember the contents offhand.

Until recently Multicore OCaml was focused on deep handlers. The people working on the formalization of effects (either for program proofs or typed effects) were quite keen to have shallow handler integrated however. Thus, the effect module of the OCaml 5 preview contains both (see https://github.com/ocaml-multicore/ocaml-multicore/blob/5.00...) since September. I fear that non-academic literature has not followed this change (on the academic side, see https://dl.acm.org/doi/10.1145/3434314 for a program proofs point of view).

Yes, it's announcing that the next but one version, 5.0, will support multicore and effect handlers.

For what it's worth you can actually start using Multicore OCaml today, there are installation instructions on the wiki: https://github.com/ocaml-multicore/ocaml-multicore

ocaml-multicore/ocaml-multicore

Could you explain (in simple terms if possible) how the Multicore OCaml achieves a memory model which is much simpler on more efficient than in Java or C (mentioned at https://github.com/ocaml-multicore/ocaml-multicore/wiki)?

Didn't see any mentions of critical sections (mutexes) with C++ examples in the documentation ("Bounding Data Races in Space and Time"). I'm not sure I understand the comparisons the writers are presenting.

There are getting started instructions up on https://github.com/ocaml-multicore/ocaml-multicore