asyncpg VS sqlitebrowser

- SqlAlchemy/asyncpg => you can’t use it if you’re using PgBouncer (necessary most of the time with Postgres) in transaction mode? What?? https://github.com/MagicStack/asyncpg/issues/1058

OK I stand corrected, asyncpg has these two C files:

https://github.com/MagicStack/asyncpg/blob/master/asyncpg/pr...

https://github.com/MagicStack/asyncpg/blob/master/asyncpg/pr...

If you are interested here is a post by the psycopg author about psycopg2 and 3 and performance versus asyncpg.

https://www.varrazzo.com/blog/2020/05/19/a-trip-into-optimis...

This pure Python library claims quite fabulous performance: https://github.com/MagicStack/asyncpg

I believe it because that team have done lots of great stuff but I haven't used it, I just remembered thinking it was interesting the performance was so good. Not sure how related it is to running on the asyncio loop (or which loop they used for benchmarks).

what a great post, we have had a ton of issues with users using pgbouncer and it's not because things are "broken" per se, it's just the situation is very complicated, and pgbouncer's docs are also IMO in need of updating to be more detailed and in a few critical cases less misleading, specifically the prepared statements docs.

This blog post refers to this misleading nature at https://jpcamara.com/2023/04/12/pgbouncer-is-useful.html#pre... .

> PgBouncer says it doesn’t support prepared statements in either PREPARE or protocol-level format. What it actually doesn’t support are named prepared statements in any form.

That's also not really accurate. You can use a named prepared statement just fine in transaction mode. start a transaction (so you aren't in autocommit), use a named statement, works fine. you just can't use it again in another transaction, because it will be "gone" (more accurately, "unmoored" - might be in your session, might be in someone else's session). Making things worse, when the prepared statement is "unmoored", its name can then conflict with another client attempting to use the same name.

so to use named prepared statements, you can less ideally name them with random strings to avoid conflicts, or you can DEALLOCATE the prepared statement(s) you used at the end of your transaction. for our users that use asyncpg, we have them use a uuid for prepared statements to avoid these name conflicts (asyncpg added this feature for us here: https://github.com/MagicStack/asyncpg/issues/837). however, they can just as well use DEALLOCATE ALL, set this as their `server_reset_query`, and then so that happens in transaction mode, also set `server_reset_query_always`, so that it's called at the end of transactions. Where pgbouncer here IMO entirely misleadingly documents this as "This setting is for working around broken setups that run applications that use session features over a transaction-pooled PgBouncer." - which is why nobody uses it, because pgbouncer claims this is "broken". It's not any more broken than it is to switch out the PostgreSQL session underneath a connection that uses multiple transactions. Pgbouncer can do better here and make this clearer and more accommodating of real world database drivers.

asyncpg is another great driver if you're using asyncio and want maximum performance (although they also break with DBAPI, but the tradeoff may be worth it).

asyncpg: 5.5k starts, last commit recently, ~150 issues, some incompatibility, few open PRs, extensive README. Includes benchmark showing it's supposedly 3x faster than aiopg and psycopg2, psycopg3 is not mentioned in the benchmark.

Quart-DB uses asyncpg to manage the connections and buildpg to parse the named parameter bindings.

A major performance boost specifically on inserts with timescaledb was actually starting to use https://github.com/MagicStack/asyncpg.

Simple query runs long when DB schema contains thousands of tables #186

We have a spread of different GitHub Actions based workflows that do stuff whenever a PR is proposed or merged:

https://github.com/sqlitebrowser/sqlitebrowser/tree/master/....

Most of those are oriented around building packages for various OS's (Linux, macOS, Windows) so people can try the latest code.

While there are some tests, they're more like extremely basic sanity tests and don't rely on Docker.

Those tests rely on whichever version of SQLite was downloaded and compiled into the GUI (as per above code snippet).

---

That being said, that's for the client side GUI application. There's a server side of things too (https://github.com/sqlitebrowser/dbhub.io -> dbhub.io) that does use docker for it's automated tests:

https://github.com/sqlitebrowser/dbhub.io/tree/master/.githu...

Those are integration tests though (eg "make sure we didn't bust communication with our cli", "make sure our go library still works 100% with the server"), and a reasonably decent set of End to End (E2E) tests of the web interface using Cypress.

---

Does that help? :)

Do you know about SQLite DB Browser ? It's a multi platform application that would perfectly fit your use case :).

https://sqlitebrowser.org/

Someone gave you a sqlite database. You probably started to think to install a program like sqlitebrowser, but only to view and read the data. Don't bother, you can use this extension instead.

Try https://sqlitebrowser.org/ it’s great for local use and training purposes

You can see how the db file looks for the app using https://sqlitebrowser.org/

If you want to fiddle with SQLite and don't need all the power herein, I recommend DB Browser for SQLite.

https://sqlitebrowser.org/

Hello y'all. This is my first post here. I'm learning SQL with the Northwind database. When I run a "big" query, like "SELECT * FROM Customers" the application crashes. I noticed this only happens when I run a general query, for example, if I run something more specific (and, by the hand "smaller") it doesn't happen. So, I thought it could be related to the size of the query. But I'm a total beginner so idk. It doesn't happen every time I try to use it, but still, this is frustrating. If there's no solution, what other SQL application can I use to keep learning? I found something on GitHub but this is for an older macOS version.