Our great sponsors
-
-
-
InfluxDB
Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
-
-
-
-
-
-
WorkOS
The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.
-
1. Domains are the unit of parallelism. A domain is essentially an OS thread with a bunch of extra runtime book-keeping data. You can use Domain.spawn (https://github.com/ocaml-multicore/ocaml-multicore/blob/5.00...) to spawn off a new domain which will run the supplied function and terminate when it finishes. This is heavyweight though, domains are expected to be long-running.
2. Domainslib is the library developed alongside multicore to aid users in exploiting parallelism. It supports nested parallelism and is pretty highly optimised (https://github.com/ocaml-multicore/domainslib/pull/29 for some graphs/numbers). The domainslib repo has some good examples: https://github.com/ocaml-multicore/domainslib/tree/master/te...
3. We've not tested against other forms of parallelism. There isn't anything stopping you exploiting SIMD in addition to parallelism from domains.
4. No, we've not compared performance by OS.
5. No plans for the multicore team to look at accelerator integration at the moment.
To clarify that, there are two systems here:
- domainslib schedules all tasks across all cores (like Go).
- eio keeps tasks on the same core (and you can use a shared job queue to distribute work between cores if you want).
Eio can certainly do async IO on multiple cores.
Moving tasks freely between cores has some major downsides - for example every time a Go program wants to access a shared value, it needs to take a mutex (and be careful to avoid deadlocks). Such races can be very hard to debug.
I suspect that the extra reliability is often worth the cost of sometimes having unbalanced workloads between cores. We're still investigating how big this effect is. When I worked at Docker, we spent a lot of time dealing with races in the Go code, even though almost nothing in Docker is CPU intensive!
For a group of tasks on a single core, you can be sure that e.g. incrementing a counter or scanning an array is an atomic operation. Only blocking operations (such as reading from a file or socket) can allow something else to run. And eio schedules tasks deterministically, so if you test with (deterministic) mocks then the trace of your tests is deterministic too. Eio's own unit-tests are mostly expect-based tests where the test just checks that the trace output matches the recorded trace, for example.
The Eio README has more information, plus a getting-started guide: https://github.com/ocaml-multicore/eio/blob/main/README.md
I started using OCaml in 2017, and had a rough start. But since then Dune (https://dune.build/) and the now-excellent LSP implementation have resolved the biggest issues I had. I haven't had such a pleasant development environment since Turbo Pascal! (To be fair, IntelliJ was also pretty good other than the shortcomings of Java.)
1. Domains are the unit of parallelism. A domain is essentially an OS thread with a bunch of extra runtime book-keeping data. You can use Domain.spawn (https://github.com/ocaml-multicore/ocaml-multicore/blob/5.00...) to spawn off a new domain which will run the supplied function and terminate when it finishes. This is heavyweight though, domains are expected to be long-running.
2. Domainslib is the library developed alongside multicore to aid users in exploiting parallelism. It supports nested parallelism and is pretty highly optimised (https://github.com/ocaml-multicore/domainslib/pull/29 for some graphs/numbers). The domainslib repo has some good examples: https://github.com/ocaml-multicore/domainslib/tree/master/te...
3. We've not tested against other forms of parallelism. There isn't anything stopping you exploiting SIMD in addition to parallelism from domains.
4. No, we've not compared performance by OS.
5. No plans for the multicore team to look at accelerator integration at the moment.
For both 2 and 3 you may find https://github.com/ocaml/RFCs/blob/unboxed-types/rfcs/unboxe... very interesting. I'm hoping that progresses beyond the RFC stage.
If you start a project today I would really try to use esy (https://esy.sh/)
I actually don’t use it myself but it seems to bring the modern programming language experience to OCaml