Show HN: prometeo – a Python-to-C transpiler for high-performance computing

• prometeo

  10 610 0.0 Python

  An experimental Python-to-C transpiler and domain specific language for embedded high-performance computing

This is awesome! The direction of using a subset of python, while leveraging the user base and static typing to accomplish some other everyday task in a different language is very legit IMO.

I took a cursory look at:

https://github.com/zanellia/prometeo/blob/master/prometeo/cg...

It seems quite similar in spirit to

https://github.com/adsharma/py2many/blob/main/pyrs/transpile...

I'm not spending much time on py2many last few months (started a new job). Let me know if any of it sounds useful - especially the ability to transpile to 7-8 languages including Julia, C++ and Rust.

• acados

  5 674 8.7 C

  Fast and embedded solvers for nonlinear optimal control

  

> Soo... it takes Python syntax and produces a C program, with no links back to Python - is that right? It uses a strict subset of Python, so that Prometeo programs are valid Python, but not necessarily the opposite. Is that fair?

yep

> Do you envisage this being a conduit for tight loop optimisation in Python? Or is it rather "you'd like a C program but can't write C good"?

There are already plenty of options for calling high-performance libraries from Python. Now 1) interpreting Python programs that use, e.g., NumPy, can be slow. 2) Compiling these programs using, e.g., Cython or Nuitka, can speed up the code across calls to high-performance libraries, but the resulting code will still rely on the Python runtime library, which can be slow/unreliable in an embedded context.

Coming to the second part of the question, writing C code directly is definitely an option, but, after doing a bit of that, I realized how tedious/error prone it is to develop/maintain/extend relatively complex code bases for embedded scientific computing (e.g. this one https://github.com/acados/acados). Or, to put it as Bjarne Stroustroup once said "fiddling with machine addresses and memory is rather unpleasant and not

• InfluxDB

  www.influxdata.com sponsored

  Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

  

• blasfeo

  6 276 5.3 Assembly

  Basic linear algebra subroutines for embedded optimization

Intrinsics (or directly assembly) are used in BLASFEO (https://github.com/giaf/blasfeo) the linear package used by prometeo. It would be cool to generate assembly directly for a few things, but that would require quite a bit of work!

• hpipm

  2 473 7.7 C

  High-performance interior-point-method QP and QCQP solvers

If somebody is interested to use this prometeo tool to implement a MPC controller, would be possible to interface it directly also to HPIPM (https://github.com/giaf/hpipm) or similar standalone, well optimized QP solver?

• py2many

  29 590 8.1 Python

  Transpiler of Python to many other languages

  

This is awesome! The direction of using a subset of python, while leveraging the user base and static typing to accomplish some other everyday task in a different language is very legit IMO.

I took a cursory look at:

https://github.com/zanellia/prometeo/blob/master/prometeo/cg...

It seems quite similar in spirit to

https://github.com/adsharma/py2many/blob/main/pyrs/transpile...

I'm not spending much time on py2many last few months (started a new job). Let me know if any of it sounds useful - especially the ability to transpile to 7-8 languages including Julia, C++ and Rust.

• MatrixEquations.jl

  1 78 8.2 Julia

  Solution of Lyapunov, Sylvester and Riccati matrix equations using Julia

• OpenBLAS

  22 5,952 9.8 C

  OpenBLAS is an optimized BLAS library based on GotoBLAS2 1.13 BSD version.

  

I think most HPC people would disagree with this statement. State-of-the-art HPC code is still written in ASM (see e.g., https://github.com/xianyi/OpenBLAS) [that's what Intel is doing too]

• WorkOS

  workos.com sponsored

  The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.

  

• StaticCompiler.jl

  16 472 7.1 Julia

  Compiles Julia code to a standalone library (experimental)

Regarding all the questions about Julia:

There's ongoing work to reduce runtime dependencies of Julia (for example in 1.8, you can strip out the compiler and metadata), but then it's only approaching Go/Swift and other static languages with runtimes.

Generating standalone runtime free LLVM is another path, that is actually already pretty mature as it's what is being done for the GPU stack.

Someone just has to retarget that to cpu LLVM, and there's a start here: https://github.com/tshort/StaticCompiler.jl/issues/43

• Metatheory.jl

  5 332 8.1 Julia

  General purpose algebraic metaprogramming and symbolic computation library for the Julia programming language: E-Graphs & equality saturation, term rewriting and more.

  

Well IMO it can definitely be rewritten in Julia, and to an easier degree than python since Julia allows hooking into the compiler pipeline at many areas of the stack. It's lispy an built from the ground up for codegen, with libraries like (https://github.com/JuliaSymbolics/Metatheory.jl) that provide high level pattern matching with e-graphs. The question is whether it's worth your time to learn Julia to do so.

You could also do it at the LLVM level: https://github.com/JuliaComputingOSS/llvm-cbe

For interesting takes on that, you can see https://github.com/JuliaLinearAlgebra/Octavian.jl which relies on loopvectorization.jl to do transforms on Julia AST beyond what LLVM does. Because of that, Octavian.jl beats openblas on many linalg benchmarks

• llvm-cbe

  14 791 6.5 C++

  resurrected LLVM "C Backend", with improvements

  

Well IMO it can definitely be rewritten in Julia, and to an easier degree than python since Julia allows hooking into the compiler pipeline at many areas of the stack. It's lispy an built from the ground up for codegen, with libraries like (https://github.com/JuliaSymbolics/Metatheory.jl) that provide high level pattern matching with e-graphs. The question is whether it's worth your time to learn Julia to do so.

You could also do it at the LLVM level: https://github.com/JuliaComputingOSS/llvm-cbe

For interesting takes on that, you can see https://github.com/JuliaLinearAlgebra/Octavian.jl which relies on loopvectorization.jl to do transforms on Julia AST beyond what LLVM does. Because of that, Octavian.jl beats openblas on many linalg benchmarks

• Octavian.jl

  17 222 3.9 Julia

  Multi-threaded BLAS-like library that provides pure Julia matrix multiplication

  

Well IMO it can definitely be rewritten in Julia, and to an easier degree than python since Julia allows hooking into the compiler pipeline at many areas of the stack. It's lispy an built from the ground up for codegen, with libraries like (https://github.com/JuliaSymbolics/Metatheory.jl) that provide high level pattern matching with e-graphs. The question is whether it's worth your time to learn Julia to do so.

You could also do it at the LLVM level: https://github.com/JuliaComputingOSS/llvm-cbe

For interesting takes on that, you can see https://github.com/JuliaLinearAlgebra/Octavian.jl which relies on loopvectorization.jl to do transforms on Julia AST beyond what LLVM does. Because of that, Octavian.jl beats openblas on many linalg benchmarks

• artiq

  2 403 9.6 Python

  A leading-edge control system for quantum information experiments

  

No, I mean nanosecond and picosecond precision real-time systems. Exhibit A: https://github.com/m-labs/artiq

Suggest a related project