No-GIL mode coming for Python

This page summarizes the projects mentioned and recommended in the original post on news.ycombinator.com

Our great sponsors
  • InfluxDB - Power Real-Time Data Analytics at Scale
  • WorkOS - The modern identity platform for B2B SaaS
  • SaaSHub - Software Alternatives and Reviews
  • scala-cli

    Scala CLI is a command-line tool to interact with the Scala language. It lets you compile, run, test, and package your Scala code (and more!)

  • The new official Scala build tool / compiler front end (scala-cli) is amazing,

    https://scala-cli.virtuslab.org/

    The thing that really struck me after years of python is how it lets you out dependencies directly in a comment on top of a script and it will download and run with them automatically, without poisoning any system settings. It's so simple!

  • problems

    Discussions about problems with the current C Api (by capi-workgroup)

  • > Does that stay true once the GIL turns back on?

    In the current nogil implementation, AFAICS, it seems the GIL can't be turned back on so there is no answer yet.

    Theoretically, you could have a one-off operation which fixes all objects when the GIL is turned on. However, there's no way to get all objects in Python. gc.get_objects() only returns tracked objects, and there is no way to list untracked objects.

    It seems, three fields are exposed on every CPython object in the stable ABI, any change which affects their offsets will break the stable ABI. https://github.com/capi-workgroup/problems/issues/4#issuecom...

    > I'm not following why this affects ABI compatibility, sorry.

    True, PyType_FromSpec can set tp_alloc to a wrapper function which papers over the difference in what the "allocfunc" should initialise the memory to.

    Re performance - merging the change is the only way people will actually start targeting nogil.

  • InfluxDB

    Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

    InfluxDB logo
  • nix

    Nix, the purely functional package manager

  • The old-style nix-shell command in Nix can do this[1] for every language and package Nixpkgs supports, although it’s not that often used because it ties your shebangs to Nix. (An equivalent feature for the new CLI is a work in progress[2].)

    [1] https://nixos.org/manual/nix/stable/command-ref/nix-shell.ht...

    [2] https://github.com/NixOS/nix/pull/5189

  • cloudpickle

    Extended pickling support for Python objects

  • I believe you just pass objects instead, like you would in OOP, and take the hit of pickling and unpickling them every time.

    If you really want to pass lambdas, you can use a third party library to pickle them

    https://github.com/cloudpipe/cloudpickle

    Yes, this is not great.

  • CPython

    The Python programming language

  • > I'm going to miss the thread-safety non-GIL python offered.

    The old method — which is the GIL, or non-non-GIL — provides no thread safety to Python code. It only protects C code

    > Can we get an in-python fork() mechanic ... please?

    You probably want Multiple Subinterpreters: https://github.com/python/cpython/issues/84692

  • extrainterpreters

    Utilities for using Python's PEP 554 subinterpreters

  • It's basically becoming reality already, PEP 684 per-interpreter gil is the required structure for this (coming in Python 3.12), now only interface on top of that remains, see WIP like https://github.com/jsbueno/extrainterpreters the full interface to gil-free interpreter threads is coming in

  • AspNetCoreDiagnosticScenarios

    This repository has examples of broken patterns in ASP.NET Core applications

  • Many that praise async/await in C#, kind of forget it took about 10 years to spread across all the layer of the language and runtime, since it was done via IL rewriting, it caused several issues with F# async tasks, due to the age of the ecosystem plenty of code isn't async/await friendly and needs to be wrapped into Task.Run() or similar.

    There is a best practices guideline from one of the ASP.NET architects, https://github.com/davidfowl/AspNetCoreDiagnosticScenarios/b...

    During last year they researched adding Go/Java's approach to .NET, but now it is too late. See the ASP.NET Q&A session at BUILD 2023.

  • WorkOS

    The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.

    WorkOS logo
NOTE: The number of mentions on this list indicates mentions on common posts plus user suggested alternatives. Hence, a higher number means a more popular project.

Suggest a related project

Related posts