hpy VS psycopg2cffi

There is a merge request up to add autogen rust bindings to hpy

https://github.com/hpyproject/hpy/pull/457

Are you referencing https://github.com/hpyproject/hpy?

I do hope it takes off.

The HPy project [0] seems like a promising way out of this.

[0] https://hpyproject.org/

socketify.py breaks the record for Python no other Python WebFramework/Server as able to reach 6.2 mi requests per second before in TechEmPower Benchmarks, this puts Python at the same level of performance that Golang, Rust and C++ for web development, in fact Golang got 5.2 mi req/s in this same round. Almost every server or web framework tries to use JIT to boost the performance, but only socketify.py deliveries this level of performance, and even without JIT socketify.py is twice as fast any other web framework/server in active development, and still can be much more optimized using HPy (https://hpyproject.org/). Python will get even faster and faster in future!

I think Propose a better messaging for Python is the option and a lot of languages will learn it from Rust, because rust erros are the best described errors I see in my life lol. Cargo is amazing and I think we will need a better poetry/pip for sure, HPy project will modernize extensions and packages 📦 too https://hpyproject.org/

It also show that PyPy3 will not magically boost your performance, you need to integrate in a manner that PyPy3 can optimize and delivery CPU performance, with a more complex example maybe it can help more. But why socketify is so much faster using PyPy3? The answer is CFFI, socketify did not use Cython for integration and cannot delivery the full performance on Python3, this will be solved with HPy.

HPy integration to better support CPython, PyPy and GraalPython

Absolutely everything in CPython is a PyObject, and that can’t be changed without breaking the C API. A PyObject contains (among other things) a type pointer, a reference count, and a data field; none of these things can be changed without (again) breaking the C API.

There have definitely been attempts to modernize; the HPy project (https://hpyproject.org/), for instance, moves towards a handle-oriented API that keeps implementation details private and thus enables certain optimizations.

The only compatibility issue I've run into is database drivers.

For PostgreSQL, psycopg2 is not supported. psycopg2cffi is largely unmaintained, and the 2.9.0 version in PyPI lacks some newer features of psycopg2: the `psycopg2.sql` module and empty result sets raise a RuntimeError in Python 3.7+. The latest commit in on Github does have these changes [1]. Psycopg 3 [2] and pg8000 [3] (as user tlocke mentioned elsewhere) are viable alternates provided you aren't stuck with older versions of PostgreSQL. I'm going to continue to use psycopg2cffi until I can upgrade an old PostgreSQL 9.4 database.

For Microsoft SQL Server, pymssql does not support PyPy [4]. It's under new maintainership so it might gain support in the future. pypyodbc hasn't had any activity since 2022, and no new PyPI release since 2021 [5]. The datatypes returned can differ between libodbc1 versions. On Ubuntu 18.04 in particular: empty string columns are returned as a single space, integer columns are returned as a Decimal. Also, if you encounter a mysterious HY010 error ("Function sequence error"), you may need to upgrade libodbc1 to v2.3.7+ from v2.3.4 using the Microsoft repos.

[1]: https://github.com/chtd/psycopg2cffi

From time to time, I use pgcopy coupled with psycopg2cffi to feed large volumes of data processed by custom parsers written in Python for several formats. The whole process is 4-5x faster with PyPy.