A viable solution for Python concurrency

• numpy

  4 3 0.0

  The fundamental package for scientific computing with Python. (by colesbury)

Yikes, C extensions can't assume they are under GIL by default:

https://github.com/colesbury/numpy/commits/v1.19.3-nogil

• graalpython

  13 1,110 10.0 Python

  A Python 3 implementation built on GraalVM

  

• WorkOS

  workos.com sponsored

  The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.

  

• nogil

  31 2,853 5.7 Python

  Multithreaded Python without the GIL

- The collection thread-safety adapts some code from FreeBSD (https://github.com/colesbury/nogil/blob/nogil/Python/qsbr.c)

• go

  2,067 119,564 10.0 Go

  The Go programming language

  

- The stop-the-world implementation is influenced by Go's design (https://github.com/golang/go/blob/fad4a16fd43f6a72b6917ef...)

• gipc

  1 79 6.8 Python

  gevent-cooperative child processes and inter-process communication

This!

> gevent doesn't have good support for multiprocessing

This is where my little library https://github.com/jgehrcke/gipc might help! It's been working well for people since 2012 :)

By the way, gevent is huge for the Python ecosystem and I would love to acknowledge both Denis Bilenko and Jason Madden who have done a great job over the years. I am in particular impressed by Jason's maintenance work -- creating a new project is always exciting. Maintaining it over the years is sometimes just tedious, hard work. Jason has impressed me a lot with his one man show there.

Not sure if everybody knows, but gevent was for starters built on top of "only" libev -- another super impressive project built and maintained by practically a single person: Marc Lehmann. I learned a lot from his work, too. So so so many serious software projects build on top of libev.

Only 'recently' gevent included support for libuv, the event loop that originated from the NodeJS ecosystem.

I find it super interesting to see that CPython+gevent+libuv is highly similar to running NodeJS!

By the way, the LWN article is once again a wonderfully written piece by Jonathan Corbet. His way to doing technical writing is simply great, I have enjoyed it very much over the years and learned a lot.

• hpy

  20 1,005 8.2 Python

  HPy: a better API for Python

  

Now could be a good time to make this change, in coordination with HPy: https://github.com/hpyproject/hpy

I agree though — it’s tempting to keep extending and stretching the language to be something it was never designed for; but at some point it’s been stretched so far it loses the properties that made it attractive to start with. I like Python, but some of the things people are using it for now, they should really consider another language instead, and write a Python wrapper on top of that if they must use it from Python.

Suggest a related project