gevent VS Pyjion

A sub-question for the folks here: is anyone using the combination of gevent and PyPy for a production application? Or, more generally, other libraries that do deep monkey-patching across the Python standard library?

Things like https://github.com/gevent/gevent/issues/676 and the fix at https://github.com/gevent/gevent/commit/f466ec51ea74755c5bee... indicate to me that there are subtleties on how PyPy's memory management interacts with low-level tweaks like gevent that have relied on often-implicit historical assumptions about memory management timing.

Not sure if this is limited to gevent, either - other libraries like Sentry, NewRelic, and OpenTelemetry also have low-level monkey-patched hooks, and it's unclear whether they're low-level enough that they might run into similar issues.

For a stack without any monkey-patching I'd be overjoyed to use PyPy - but between gevent and these monitoring tools, practically every project needs at least some monkey-patching, and I think that there's a lack of clarity on how battle-tested PyPy is with tools like these.

I'm not sure how much it replicates the CSP model, but the closest thing I've found to Go-style concurrency in Python is gevent: https://github.com/gevent/gevent

I personally still prefer to use it in all my projects.

Greenlet seems to have some troubles compiling with Python 3.9. https://github.com/gevent/gevent/issues/1627

It exists, was created by microsoft employees, and is referenced in the article: https://www.trypyjion.com/

I've actually come across and started using Pyjion recently (https://github.com/tonybaloney/pyjion); how does Pypy compare, both in terms of performance and purpose? There seems to be a lot of overlap...

Python is capable of doing things at runtime that are really hard to statically compile around, such as monkeypatching methods onto existing objects. You can compile it, but it's complicated. One strategy is to use a JIT that can observe application state at runtime and then invalidate code as it becomes obsoleted by changes, but it's complicated. See pyjion for an example.

When I say "Python" I am referring to the standard CPython interpreter which most people use. But there is also PyPy, which includes a Just In Time compile that compiles selected code into machine language on the fly, as needed. pyjion is another JIT compiler that generates machine language on the fly, and you can install it with pip. Or you could work for Facebook and use Cinder. Cython, Nuitka and Pyston are other alternatives.

and if you need more speed you can try https://www.pypy.org/ or https://github.com/tonybaloney/Pyjion or https://www.pyston.org/

Finally, there is also Pyjion which based on its website is “A drop-in JIT Compiler for Python 3.10” (https://www.trypyjion.com/). We will be covering it on a separate writeup. See you next time ;-).

Though some exciting stuff happening over the next few years, Python is getting faster, has been for awhile, and stuff like Pyjion https://www.trypyjion.com/, a drop in C# powered JIT compiler is starting to approach usable. Rust and Python seem to be best buds right now, so more extension libraries in rust, a newer more approachable language than say C/C++ but with a similar speed. Sign me up!

Pyjion Easy to use, small compiler. Increase performance of our 🐌 CPython.