aiosql VS SQLAlchemy

> resort to raw SQL

I'm the opposite, I would rather write SQL than "resorting to" ORM queries, which is why my favourite libraries are aiosql[1] in Python, Hugsql[2] in Clojure and similar: write the queries as SQL in .sql files, which then get exposed as functions to your code.

[1] https://nackjicholson.github.io/aiosql/

[2] https://www.hugsql.org/

I prefer to use aiosql https://nackjicholson.github.io/aiosql/ to organize my SQL and have it in a SQL folder. It looks like this where colons specify variables:

I tend to do a lot of data-heavy projects, so I tend to eschew ORM-style code and use a project called aiosql to bind raw SQL to python methods, and offload as much expensive computation to the DB as possible. If I'm prototyping an endpoint (e.g. calculating percentiles for some midsized time-series data), and just need a non-performant working placeholder, it's extremely easy to dump a SQL table to pandas and yeet something together in a few lines - then smoothly replace it with a more performant SQL query down the road. Highly contextual move, but I find it to be an awesome balancing point between flexibility, scalability, performance, productivity, etc.

As one of the rare Python developers who actually like SQL, my favourite database library is aiosql

Only slightly off-topic, poking around in there led me to aiosql, which takes an idea I'd had and jumps forward a good long way. :-)

I've been doing something in this vein for a big personal project, using this python library: https://nackjicholson.github.io/aiosql/.

In short, I'm using a run of the mill stack (Caddy/Gunicorn/Flask/Postgres) - but with the twist that all my core logic is defined in plaintext SQL files, which get bound into namespaced Python methods by aiosql. Routing, error handling, templating, etc. are all done in Python - but all data manipulation and processing are outsourced to the DB level. All database object definitions are laid out in a massive, idempotent "init_db" method that gets called at launch, so I can essentially point the app at a fresh instance of Postgres and rebuild from scratch. The design is primarily driven by my personal distaste for ORMs, but I've found it extremely beneficial in terms of rigid typing, integrity checks, and performance.

Definitely check out https://nackjicholson.github.io/aiosql/ if you want to stick with SQL

I took that approach on my latest Flask project and it’s gone quite swimmingly. The problem I ran into was that a lot of the ecosystem, and therefore documentation, blog posts, helper libraries, etc., are all written under the assumption that you’re using an ORM. It took a while to figure out how to work around that, but once I did, I was home clear.

I also used a helper library to automatically map namespaced .sql files onto python functions with various return types, which made the development process way more elegant: https://nackjicholson.github.io/aiosql/. Absolute game changer if you plan to go this route - can’t recommend it highly enough.

I've been using FastAPI for some time, and now I'm using it as a full web framework (not just for REST APIs). I like writing SQL without ORMs, so the combination of aiosql[0] + FastAPI + Jinja2 works great. Add HTMX[1] and even interactive websites become easy.

That's in fact the stack I am using to build https://drwn.io/ and I couldn't enjoy it more.

Thanks Sebastián for creating it!

[0] https://github.com/nackjicholson/aiosql

In SQLAlchemy, models representing database tables typically subclass sqlalchemy.orm.DeclarativeBase (this class supersedes the sqlalchemy.orm.declarative_base function). Accordingly, the abstract base class in this database wrapper component is a sqlalchemy.orm.DeclarativeBase subclass, accompanied by another custom base class providing additional dunder methods.

OK -

can we start considering binary files committed to a repo, even as data for tests, to be a huge red flag, and that the binary files themselves should instead be generated at testing time by source code that's stated as reviewable cleartext. This would make it much harder (though of course we can never really say "impossible") to embed a substantial payload in this way.

when binary files are part of a test suite, they are typically trying to illustrate some element of the program being tested, in this case a file that was incorrectly xz-encoded. Binary files like these weren't typed by hand, they will always ultimately come from something plaintext source.

Here's an example! My own SQLAlchemy repository has a few binary files in it! https://github.com/sqlalchemy/sqlalchemy/blob/main/test/bina... oh noes. Why are those files there? well in this case I just wanted to test that I can send large binary BLOBs into the database driver and I was lazy. This is actually pretty dumb, the two binary files here add 35K of useless crap to the source, and I could just as easily generate this binary data on the fly using a two liner that spits out random bytes. Anyone could see that two liner and know that it isn't embedding a malicious payload.

If I wanted to generate a poorly formed .xz file, I'd illustrate source code that generates random data, runs it through .xz, then applies "corruption" to it, like zeroing out the high bit of every byte. The process by which this occurs would be all reviewable in source code.

Besides, flama also provides support for SQL databases via SQLAlchemy, an SQL toolkit and Object Relational Mapper that gives application developers the full power and flexibility of SQL. Finally, flama also provides support for HTTP clients to perform requests via httpx, a next generation HTTP client for Python.

Alembic is a lightweight database migration tool for usage with SQLAlchemy. The term migration can be a little misleading, because in this context it doesn't mean to migrate to a different database in the sense of using a different version or a different type of database. In this context, migration refers to changes to the database schema: add a new column to a table, modify the type of an existing column, create a new index, etc..

To simplify this process, Cerbos developers have come up with adapters for popular Object-Relational Mapping (ORM) frameworks. You can check out for more details on the query plan repo - which also contains adapters for Prisma and SQLAlchemy - as well as a fully functioning application using Mongoose as its ORM.

That above pattern is one I've seen people do even recently, using the "select().c" attribute which from very early versions of SQLAlchemy is defined as "the columns from a subquery of the SELECT" ; this usage began raising deprecation warnings in 1.4 and is fully removed in 2.0 as it was a remnant of a much earlier version of SQLAlchemy. it will do exactly as you say, "make a subquery for each filter condition".

the moment you see SQLAlchemy doing something you see that seems "asinine", send an example to https://github.com/sqlalchemy/sqlalchemy/discussions and I will clarify what's going on, correct the usage so that the query you have is what you expect, and quite often we will add new warnings or documentation when we see people doing things we didn't anticipate.

The creator and lead maintainer of SQLAlchemy, one of the most popular and most used Python library for accessing databases (who doesn't?) gave a rather interesting response to PEP703.

If this doesn't ring any alarm bells I don't know what will.

> Basically for the moment the GIL-less idea would likely be burdensome for us and the fact that it's only an "option" seems to strongly imply major compatibility issues that we would not prefer.

https://github.com/sqlalchemy/sqlalchemy/discussions/10002#d...

Recently I discovered BigQuery public datasets - just over 200 datasets available for directly querying via SQL. I think this is a great thing! I can connect these direct to an analytics platform (we use Apache Superset which uses Python SQLAlchemy under the hood) for example and just start dashboarding.

When using python with sql databases like postgres or mariadb or SQLite you would use SQLAlchemy or another ORM of if you're feeling brave, you code it by hand. With ORMs you provide the address of your database and it connects for you, letting you use abstractions instead of writing all the SQL yourself (kind of analogous to using vlookups or index match instead of manually entering data).