cinder VS mypyc

They’ve kind of done that with Cinder https://github.com/facebookincubator/cinder. Threads and Instagram lie on top of this and it’s essentially a fork of Python 3.10.

I think the problem there is even a company like Meta wants to rest on the shoulders of the giant Python community. It makes sense for Meta to try to get parts of Cinder into Python.

I used to work on the Cinder JIT and can help document any passes you find interesting or confusing.

Have a look at Cinder - https://github.com/facebookincubator/cinder - it's Meta's performance oriented fork of CPython that they use to run Instagram (which is a big Django app).

You might enjoy Cinder then. It's based on CPython so it is nearly 100% compatible.

https://github.com/facebookincubator/cinder/

Disclaimer: I used to work on it.

I think Cinder's Static Python, which also performs runtime type checking, is more ambitious. Though it's not production ready yet.

Here's a fork that implements that https://github.com/facebookincubator/cinder - it might be nice to one day get that up streamed but obviously it'll be controversial and it certainly needs more time to bake. Hopefully at some point we can make it a pip installable extension though.

The other thing that was interesting to me, was the potential of type annotations to help make for a faster, safer experience on the compiler end of things. One example is seen in Meta’s Cinder project, on the docs it explains how typing can be used to reduce the number of steps for the compiler ([cinder/static_python.rst at cinder/3.8 · facebookincubator/cinder · GitHub](https://github.com/facebookincubator/cinder/blob/cinder/3.8/CinderDoc/static_python.rst)), making it more effective.

Instagram is a huge Django app and unlike all the ML libraries, Django isn't highly optimized low-level code wrapped in Python, at least not by default. That's why Facebook are putting so much work into running a more performant fork of CPython - https://github.com/facebookincubator/cinder

That would be interesting! You might already be aware. But there's mypyc[0], which is an AOT compiler for Python code with type hints (that, IIRC, mypy uses to compile itself into a native extension).

Wanted to give you a head-start on the lit-review for your students I guess :)

[0] https://github.com/mypyc/mypyc

https://github.com/mypyc/mypyc

> Mypyc compiles Python modules to C extensions. It uses standard Python type hints to generate fast code. Mypyc uses mypy to perform type checking and type inference.

> Mypyc can compile anything from one module to an entire codebase. The mypy project has been using mypyc to compile mypy since 2019, giving it a 4x performance boost over regular Python.

I have not experience a 4x boost, rather between 1.5x and 2x. I guess it depends on the code.

Funny how emergence works with tools. Give a language too few tools but viral circumstances - the ecosystem diverges (Lisps, Javascript). Give it too long an iteration time but killer guarantees, you end up with committees. Python not falling into either of these traps should be understood as nothing short of magic in emergence.

I only recently discovered that python's reference typechecker, mypy, has a small side project for typed python to emit C [1], written entirely in python. Nowadays with python's rich specializer ecosystem (LLVM, CUDA, and just generally vectorized math), the value of writing a small program in anything else diminishes quickly.

Imagine reading the C++wg release notes in the same mood that you would the python release notes.

[1] https://github.com/mypyc/mypyc

> Note that the mypyc issue tracker lives in this repository! Please don't file mypyc issues in the mypy issue tracker.

See https://github.com/mypyc/mypyc/blob/master/show_me_the_code....

What's the difference with mypyc [0] ? It also compiles Python to native code.

[0]: https://github.com/mypyc/mypyc

More recent approaches include mypyc which is (on the tin) quite close to what you describe, and taichi that lives in between.

https://github.com/mypyc/mypyc

> Mypyc compiles Python modules to C extensions. It uses standard Python type hints to generate fast code. Mypyc uses mypy to perform type checking and type inference.

Regarding the large execution time mentioned in your article, I discovered (mypyc)[https://github.com/mypyc/mypyc] on this subreddit in a post from the black formatter team https://www.reddit.com/r/Python/comments/v2009i/im_that_person_who_got_black_compiled_with_mypyc/?utm_medium=android_app&utm_source=share

It's still alpha, but mypyc lets ordinary Python code be compiled to C extensions just by using type hints from the standard lib. It's currently used in the mypy project.

you can check out [`mypyc`](https://github.com/mypyc/mypyc) . It is used by mypy, black for speedup