hypopg VS ruby-pg

It works particularly well with pg_stat_statements[0] which tracks execution statistics of all SQL statements executed on your Postgres database.

It leans heavily on HypoPG[1], an excellent extension to determine if PostgreSQL will use a given index without spending resources to create them.

[0] pg_stat_statements: https://www.postgresql.org/docs/current/pgstatstatements.htm...

[1] https://github.com/HypoPG/hypopg

This is an introduction to the hypopg PostgreSQL extension for YugabyteDB 2.15.3.0. Hypopg allows the creation of hypothetical indexes, so indexes that do not really exist. This means this allows you to see what an index would do if it were created, without it actually being created, and therefore not influencing anything on the database. YugabyteDB 2.15.3.0 is a preview version of the YugabyteDB database.

There's a lot of good ecosystem stuff around this:

https://github.com/HypoPG/hypopg

HypoPG is a PostgreSQL extension adding support for hypothetical indexes.

An hypothetical -- or virtual -- index is an index that doesn't really exists, and thus doesn't cost CPU, disk or any resource to create. They're useful to know if specific indexes can increase performance for problematic queries, since you can know if PostgreSQL will use these indexes or not without having to spend resources to create them.

With one approach to using it here: https://www.percona.com/blog/2019/07/22/automatic-index-reco...

If you installed Postgres via Homebrew, you need to configure bundler so that when it installs the pg gem, it knows where to find the pg_config executable, which is installed as part of Postgres. The pg gem is the Ruby interface to Postgres and requires pg_config during installation. We can use this command to configure bundler so that it can find it and successfully install pg.

Before we proceed, are you aware that a lot of popular database drivers for Ruby (and Python? not sure) implement the performance-critical bits in good old natively compiled C?

For example, the Ruby postgres gem: https://github.com/ged/ruby-pg/tree/master/ext

(I wasn't sure until I checked just now, so I'm not questioning your familiarity with the tech. Just not sure if that's commonly known)

    So no, it's not the database, it's your interpreted language.

$ gem info pg *** LOCAL GEMS *** pg (1.3.4) Authors: Michael Granger, Lars Kanis Homepage: https://github.com/ged/ruby-pg License: BSD-2-Clause Installed at: /Users/jvon1904/.rvm/gems/ruby-3.0.3 Pg is the Ruby interface to the PostgreSQL RDBMS

The low level pg gem has support for Postgres streaming, but this is row-based. You're wanting to stream effectively from a single field, which Postgres won't do for you.

The ruby-pg gem provides an interface between your Ruby code and your PostgreSQL database.

set RUBY_DLL_PATH=/bin as in our CI before running the ruby app.

> each is in its own OS thread and GVL releases when the thread blocks.

The GVL isn't automatically released when a thread blocks on IO. Each bit of native code performing IO has to explicitly release it like in the pg gem here: https://github.com/ged/ruby-pg/blob/fb465855ce1dd12cf7eb69c9...