py2many VS PyO3

> Another problem is that there are hundreds of built-in library functions that need to be compiled from Python from C

An approach I've advocated as one of the main authors of py2many is that all of the python builtin functions be written in a subset of python[1] and then compiled into native code. This has the benefit of avoiding GIL, problems with C-API among other things.

Do checkout the examples here[2] which work out of the box for many of the 8-9 supported backends.

[1] https://github.com/py2many/py2many/blob/main/doc/langspec.md

https://github.com/py2many/py2many/blob/main/doc/langspec.md

Reimplement a large enough, commonly used subset of python stdlib using this dialect and we may be in the business of writing cross platform apps (perhaps start with android and Ubuntu/Gnome)

For py2many, there is an informal specification here:

https://github.com/py2many/py2many/blob/main/doc/langspec.md

Would be great if all the authors of "python-like" languages get together and come up with a couple of specs.

I say a couple, because there are ones that support the python runtime (such as cython) and the ones which don't (like py2many).

It'd not fully solve your issue, but have you ever seen https://github.com/py2many/py2many ?

Cython doesn't consume python3 type hints and needs special type hints of its own. But it's certainly more mature than other players in the field.

What we need is a rpython suitable for app programming and a stdlib written in that dialect.

https://github.com/py2many/py2many/blob/main/doc/langspec.md

No intermediate AST. To understand the various stages of transpilation and separation of language specific and independent rewriters, this file is a good starting point:

https://github.com/adsharma/py2many/blob/main/py2many/cli.py...

Link to the test that's relevant for this discussion:

https://github.com/adsharma/py2many/blob/main/tests/cases/in...

This is an explicit deviation from python's bigint, which doesn't map very well to systemsey languages. The next logical step is to build on this to have dependent and refinement types.

Work in progress here:

https://github.com/adsharma/Typpete

Integrating Rust into Python, Edward Wright, 2021-04-12 Examples for making rustpython run actual python code Calling Rust from Python using PyO3 Writing Python inside your Rust code — Part 1, 2020-04-17 RustPython, RustPython Rust for Python developers: Using Rust to optimize your Python code PyO3 (Rust bindings for Python) Musing About Pythonic Design Patterns In Rust, Teddy Rendahl, 2023-07-14

This story unfolds as a captivating journey where the agile Flounder, representing the Python programming language, navigates the vast seas of coding under the wise guidance of Sebastian, symbolizing Rust. Central to their adventure are three powerful tridents: cargo, PyO3, and maturin.

I would not recommend FFI + ctypes. Maintaining the bindings is tedious and error-prone. Also, Rust FFI/unsafe can be tricky even for experienced Rust devs.

Instead PyO3 [1] lets you "write a native Python module in Rust", and it works great. A much better choice IMO.

[1] https://github.com/PyO3/pyo3

Same w/ Rust and Python, this is really neat because now each thread could have a GIL without doing exactly what you said. The pyO3 commit to allow subinterpreters was merged 21 days ago, so this might "just work" today: https://github.com/PyO3/pyo3/pull/3446

I expected someone to write a rust-based scripting language which tightly integrated with rust itself.

In reality, it seems like the python developers and toolchain are embracing rust enough to reduce the benefits to a new alternative.

https://github.com/PyO3/pyo3

Hey HN! I am one of the people working on Bytewax. Bytewax came out of our experience working with ML infrastructure at GitHub. We wanted to use Python because we could move fast, the team was very fluent in it, and the rest of our tooling was Python-native already. We didn't want to introduce JVM-based solutions into our stack because of the lack of experience and the friction we had trying to get Python-centric tooling working with existing solutions like Flink.

In our research, we found Timely Dataflow (https://timelydataflow.github.io/timely-dataflow/, https://news.ycombinator.com/item?id=24837031) and the Naiad project (https://www.microsoft.com/en-us/research/project/naiad/) as well as PyO3 (https://github.com/PyO3/pyo3) and we thought we found a match made in heaven :). Bytewax leverages both of these projects and builds on them to provide a clean API (at least we think so) and table stakes features like connectors, state recovery, and cloud-native scaling. It has been really cool to learn about the dataflow computation model, Rust, and how to wrangle the GIL with Rust and Python :P.

Would love to get your feedback :).

`pip install bytewax` to get started. We have a page of guides (https://www.bytewax.io/guides) with ready-to-run examples.

You can practice your Rust skills by writing performant and/or gluey extensions for higher-level language such as NodeJS (checkout napi-rs) and Python or complementing JS in the browser if you target Webassembly.

For instance, checkout Llama-node https://github.com/Atome-FE/llama-node for an involved Rust-based NodeJS extension. Python has PyO3, a Rust-Python extension toolset: https://github.com/PyO3/pyo3.

They can help you leverage your Rust for writing cool new stuff.