Faster Python with Guido van Rossum

This page summarizes the projects mentioned and recommended in the original post on news.ycombinator.com

Our great sponsors
  • InfluxDB - Power Real-Time Data Analytics at Scale
  • WorkOS - The modern identity platform for B2B SaaS
  • SaaSHub - Software Alternatives and Reviews
  • cinder

    Cinder is Meta's internal performance-oriented production version of CPython. (by facebookincubator)

    One minor clarification here: Cinder[1] is actually 100% compatible with C extensions (by virtue of being a fork of CPython).

    [1]: https://github.com/facebookincubator/cinder

  • FrameworkBenchmarks

    Source for the TechEmpower Framework Benchmarks project

    In the case of the TechEmpower benchmarks, i'd argue that it's actually pretty much the same thing as real world applications: https://github.com/TechEmpower/FrameworkBenchmarks

    For example, look at a full platform Java Spring Boot example here: https://github.com/TechEmpower/FrameworkBenchmarks/tree/mast...

    The controller code, the model code and even the repository code are all very reflective of the primitives you'd find in production code.

    The only differences would be when you add additional complexity, such as validations and other code into the mix, such as additional service calls etc., but if even the simpler stuff in Python (as an example) would be noticeably slower, then it stands to reason that it wouldn't get magically faster on the basis of less overhead alone.

    The exceptions to this which i could imagine would be calls to native code, since the aforementioned Python has an amazing number crunching or machine learning support, but that is hardly relevant to web development in most cases.

    Otherwise we risk running into the "No true Scotsman" fallacy: https://en.wikipedia.org/wiki/No_true_Scotsman

    In such a case, we'd adamantly claim that these benchmarks are simply not good enough even for educated guesses and we'd have to build out our full system to benchmark it, which no one actually has the resources for - in practice you'd develop a system and then would have to do a big rewrite, as was the case with PHP and Facebook, as well as many other companies.

    On the other hand, premature optimization and dogmatic beliefs are the opposite end of the spectrum: you don't need to write everything in Rust if using Ruby would get your small application out the door in a more reasonable amount of time.

    There are no easy answers, everything depends on the context (e.g. small web dev shop vs an enterprise platform to be used by millions) and the concerns (e.g. resource usage and server budget vs being the first to market, as well as developer knowledge of tech stacks, for example market conditions of PHP vs Rust).

    Regardless, in my eyes, there's definitely a lot of value in these attempts to compare idiomatic interpretations of common logic for web development primitives (serving network requests in a known set of formats).

  • InfluxDB

    Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

  • Nim

    Nim is a statically typed compiled systems programming language. It combines successful concepts from mature languages like Python, Ada and Modula. Its design focuses on efficiency, expressiveness, and elegance (in that order of priority).

    If anybody is looking for an alternative to Python that is also a great 0->1 language but doesn't have the same wall as Python described here, check out Nim[1].

    1 - https://nim-lang.org

  • mypy

    Optional static typing for Python

    I am seeing what the GP comment says in a codebase I manage, around 47k LOC. Not massive at all, but enough where the problems mentioned above start to pop up. This is an application that runs in servers to analyze traffic data, so it has both analysis code and also a lot of code for the analysis framework. It gets hard to manage. I have unit tests and also integration tests that cover some, but not all of the code paths (it's very hard in this case to have everything covered). Most of the times, when they fail is due to something that would have been caught by static typing.

    The codebase is being slowly migrated to static typing. On one hand, as the parent says, the typing module is still immature and there are still some Python constructs (not too weird ones, see [1] for an example) that you can't type-check correctly. On the other hand, I like the fact that you can include typing slowly and not all at once, it makes the effort much easier to tackle. And, if typing works, it works well.

    Regarding performance, well. Parallelism is pretty hard to do well, and the language itself is not the fastest thing. Some parts are migrated to a C extension but that's costly in terms of development and debugging time.

    Despite all of that, I do think that Python was the best choice for our situation, and still is. Maybe from this point onwards another language would make things easier, but without Python's library support, ease and speed of development and expressiveness the cost of just getting to market would have been far higher, and probably we wouldn't have reached this point with other languages. And migrating the codebase to another language is just not worth it at all, as there are still a lot of areas we can improve quite a lot with far less effort than a full rewrite.

    1: https://github.com/python/mypy/issues/2756#issuecomment-8772...

  • nimpy

    Nim - Python bridge

    Nim can use Python's ecosystem in both directions with nimpy https://github.com/yglukhov/nimpy and nimporter https://github.com/yglukhov/nimpy

    This lets you gradually transition hot path Python modules to Nim, get compiled performance generally on par with C and Rust, whilst enjoying strong, static typing with great type inference.

    In my experience (6-7 years 4 of which are full time) Nim strikes the perfect balance of the productivity you get with Python with high performance at the same time.

    Also the metaprogramming features are incredible and, importantly, don't use a language subset but use the base Nim language itself.

  • nimporter

    Compile Nim Extensions for Python On Import!

  • status-desktop

    Status Desktop client made in Nim & QML

  • WorkOS

    The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.

  • nimqml

    Qt Qml bindings for the Nim programming language

  • skybison

    Discontinued Instagram's experimental performance oriented greenfield implementation of Python.

    Guido appears to have meant to refer to Skybison (https://github.com/facebookexperimental/skybison), which is not 100% compatible.

NOTE: The number of mentions on this list indicates mentions on common posts plus user suggested alternatives. Hence, a higher number means a more popular project.

Suggest a related project

Related posts