Polyphony VS trio

When using green threads/fibers/coroutines, an interesting technique to make signal handling safer is to run the signal handler asynchronously on a separate fiber/green thread. That way most of the problems of dealing with signals go away, and there's basically no limitation on what you can do inside the signal handler.

I've successfully used this technique in Polyphony [1], a fiber-based Ruby gem for writing concurrent programs. When a signal occurs, Polyphony creates a special-purpose fiber that runs the signal handling code. The fiber is put at the head of the run queue, and is resumed once the currently executed fiber yields control.

[1] https://github.com/digital-fabric/polyphony

Read the release notes here: https://github.com/orgs/digital-fabric/discussions/110 The Polyphony docs: https://www.rubydoc.info/gems/polyphony The Polyphony repository: https://github.com/digital-fabric/polyphony

Polyphony is a gem for building highly-concurrent Ruby programs. It utilizes Ruby fibers to provide a high-performance safe environment for launching any number of concurrent operations. Under the hood, Polyphony employs io_uring to maximize I/O performance (libev is used on platforms other than recent Linux kernels).

Is monkey-patching inherently bad? Should its use make Polyphony illegitimate? These are the questions I'm exploring in my latest article.

how is this diff from: https://github.com/digital-fabric/polyphony

Infinite loops are great for expressing long-running concurrent operations. In this article I’ll discuss the use of infinite loops as a major construct when writing concurrent apps in Ruby using Polyphony. I’ll show how infinite loops differ from normal, finite ones; how they can be used to express long-running tasks in a concurrent environment; and how they can be stopped. Read it now!

Polyphony is a library for writing highly concurrent Ruby apps. Polyphony harnesses Ruby fibers and a powerful io_uring-based I/O runtime to provide a solid foundation for building high-performance concurrent Ruby apps.

> Curious if there’s anything public about improving ruby performance from the I/O angle mentioned in the post.

I'm currently working on Polyphony [0], a Ruby gem for writing highly-concurrent Ruby apps. It uses Ruby fibers under the hood, and does I/O using io_uring (on Linux, there is also a libev-based backend).

[0] https://github.com/digital-fabric/polyphony

Incidentally, the alternative event loop implementation trio in python does not have "gather", you also need channels, and it's a deliberate design choice - there is some discussion about that in this ticket https://github.com/python-trio/trio/issues/2188

trio – a friendly Python library for async concurrency and I/O

trio. the best code, the best documentation, awesome community.

I'll +1 the Trio shoutout [1], but it's worth emphasizing that the core concept of Trio (nurseries) now exists in the stdlib in the form of task groups [2]. The article mentions this very briefly, but it's easy to miss, and I wouldn't describe it as a solution to this bug, anyways. Rather, it's more of a different way of writing multitasking code, which happens to make this class of bug impossible.

[1] https://github.com/python-trio/trio

[2] https://docs.python.org/3/library/asyncio-task.html#task-gro...

It's similar to manual memory management.

Structured concurrency is one approach to solving this problem. In a structured concurrency a promise would not go out of scope unhandled. Not sure how you would add APIs for it though.

See Python's trio nurseries idea which uses a python context manager.

https://github.com/python-trio/trio

I'm working on a syntax for state machines and it could be used as a DSL for promises. It looks similar to a bash pipeline but it matches predicates similar to prolog.

In theory you could wire up a tree of structured concurrency with this DSL.

https://github.com/samsquire/ideas4#558-assign-location-mult...

Not complete - doesn't include Task Groups [1]

In fairness they were only included in asyncio as of Python 3.11, which was released a couple of weeks ago.

These were an idea originally from Trio [2] where they're called "nurseries" instead of "task groups". My view is that you're better off using Trio, or at least anyio [3] which gives a Trio-like interface to asyncio. One particularly nice thing about Trio (and anyio) is that there's no way to spawn background tasks except to use task groups i.e. there's no analogue of asyncio's create_task() function. That is good because it guarantees that no task is ever left accidentally running in the background and no exception left silently uncaught.

[1] https://docs.python.org/3/library/asyncio-task.html#task-gro...

[2] https://github.com/python-trio/trio

[3] https://anyio.readthedocs.io/en/latest/