pgvector VS postgres

# Install Postgres and pgvector !apt-get update && apt install postgresql postgresql-server-dev-14 !git clone --branch v0.6.2 https://github.com/pgvector/pgvector.git !cd pgvector && make && make install # Start database !service postgresql start !sudo -u postgres psql -U postgres -c "ALTER USER postgres PASSWORD 'pass';"

When talking about Vector Databases, in the market we can find the specialized ones and multi-model, most of the major database providers like Oracle, PostgreSQL or MongoDB, for mention some of them, have integrated a specific solution to retrieve vector data.

For this example, I wanted to focus on how pgvector  – an open-source vector similarity search for Postgres – can be used to identify data similarities that exist in enterprise data.

pgvector supports dense vector search well, but it does not have plan to support sparse vector.

There's an issue in the pgvector repo about someone having several ~10-20million row tables and getting acceptable performance with the right hardware and some performance tuning: https://github.com/pgvector/pgvector/issues/455

I'm in the early stages of evaluating pgvector myself. but having used pinecone I currently am liking pgvector better because of it being open source. The indexing algorithm is clear, one can understand and modify the parameters. Furthermore the database is postgresql, not a proprietary document store. When the other data in the problem is stored relationally, it is very convenient to have the vectors stored like this as well. And postgresql has good observability and metrics. I think when it comes to flexibility for specialized applications, pgvector seems like the clear winner. But I can definitely see pinecone's appeal if vector search is not a core component of the problem/business, as it is very easy to use and scales very easily

The Spring AI project aims to streamline the development of applications that incorporate artificial intelligence functionality without unnecessary complexity. On this example we use features like: Embedding, Prompts, ETL and save all embedding on PGvector(Postgres Vector database)

Official GitHub repository of the pgvector extension

pgvector 0.6.0 was just released and will be available on Supabase projects soon. Again, a special shout out to Andrew Kane and everyone else who worked on parallel index builds.

The pgvector extension adds vector similarity search capabilities to your PostgreSQL database. To use the extension, you have to first create it in your database. You can install the extension, by connecting to your database and running the CREATE EXTENSION command from the psql command prompt:

- Now, the connection between our API servers and the database was slow (a few hundred ms per query), so we moved to self-hosting Postgres which was pretty painful. We tried to use https://github.com/supabase/postgres, but the documentation was very lacking and we had to make a bunch of modifications to get it to work. After we got it working, it was pretty smooth though - pretty easy to implement backups, etc.

I think Supabase generally does good work, but I don't think they can be given credit for pgvector, if that's what you're indicating (I might have misread).

As I understand, Andrew Kane is the principal author of pgvector, and has worked on it for almost two years before Supabase added support for it.

See also https://github.com/pgvector/pgvector/issues/54 and https://github.com/supabase/postgres/pull/472.

we merged the pgvector PR about 2 weeks ago (https://github.com/supabase/postgres/pull/472). If you're missing anything for your CLI don't hesitate to reach out and we'll see if we can integrate it into the product (my email is in my profile)

as an aside, Pinecone looks great

One of the things they recommend is separating the PostgreSQL DB from the rest of the stack. And they also provide an ansible playbook to set up a postgres DB: https://github.com/supabase/postgres/blob/625899e687047a9da658f3f8cc6dd91ac9769694/ansible/playbook.yml

> entire solution from docker

We PR'd this into our docker-compose today [0]. We're always a bit slammed during Launch Week, so if you spot any problems let use know and we'll patch it up asap.

The extension is also deployed directly into our PG bundle [1] which is available in docker [2]

> The Gui for adding roles and tying them to postgres access is very slick with hasura. Is this done manually via SQL commands with supabase?

I haven't tried Hasura so I don't know if this is a direct comparison. pg_graphql works with Postgres Row Level Security - we provide a GUI for this in our Dashboard, but they are also just native PG Policies, so you can write them in raw SQL

[0] https://github.com/supabase/supabase/pull/6138/files#diff-41...

[1] https://github.com/supabase/postgres

[2] https://hub.docker.com/r/supabase/postgres

and can also PR if it's something useful for everyone: https://github.com/supabase/postgres

Show HN: Postgres Docker image with common extensions\ (23 comments)

Hi! I'm one of the contributors to the repo. Just to clarify, our Docker image [0] only contains the latest version of Postgres (13) and the common extensions listed out here [1]. All the other features such as this [2] and this [3] are only available in the AWS EC2 or DO droplet images. We've since updated our README to make that clearer :-)

You can still connect the DB with a PgBouncer image spun up in another container however. Unfortunately, I can't really recommend you which one since there doesn't seem to be an official Docker image for PgBouncer and I myself have never tried any of the existing ones out there. If you're looking to use PostgREST however, they do have an official Docker image that you can use over here [4].

[0]: https://hub.docker.com/r/supabase/postgres

[1]: https://github.com/supabase/postgres#extensions

[2]: https://github.com/supabase/postgres#enhanced-security

[3]: https://github.com/supabase/postgres#additional-goodies

[4]: https://hub.docker.com/r/postgrest/postgrest/