elasticsearch-py VS pg_similarity

FD: I have a friend who works at Elastic, though he doesn't really colour my opinions of things.

> Firstly, dick moves like this: https://github.com/elastic/elasticsearch-py/pull/1623

I understand that this is unpopular, but you can make a very strong argument that it's to prevent weird errors in the future. I'm also guilty of littering my code with Asserts to ensure the universe is working fine.

The alternative is to allow it to work and then you end up with weird issues like when you connect mysql client to mariadb server (and vice-versa): https://stackoverflow.com/questions/50169576/mysql-8-0-11-er...

> Secondly, I don't buy the argument from Elastic any more. Yes, the ethical thing to do when you're making money from someone's work is at least contribute back. At the same time though, they're making money from packaging it up and selling it _as a service_. That "as a service" part is where they're making the bucks.

That's just an opinion, yes they have a service, and yes it competes with Amazon. Is it cool for Amazon to take a body of work and sell it without supporting it? Are amazon actually supporting it? Is it the same as Elastic using Lucene? (not really because Elastic submits a the majority of fixes to Lucene, but, you get it).

it's kinda gray, I'm sure Amazon thinks they're the good guy, but it's hard for me to look at Elastic as the bad guy in all this.

Oh man, https://github.com/elastic/elasticsearch-py/issues/1734 is a disappointing read. I know ES wants to save their business, but alienating users isn't exactly the path to success.

For Postgres, there is an extension that provides that.

Note that pgvector isn't supported on any of the large cloud providers' hosted Postgres offerings, other than Supabase. https://github.com/pgvector/pgvector#hosted-postgres has instructions on how to add your voice to request it to be added!

(It does seem that the ancient https://github.com/eulerto/pg_similarity is supported by RDS and Google Cloud - but it's hard to tell whether attention was paid to its performance characteristics with nearly the rigor that pgvector seems to have been designed.)

While I don't doubt that you know your usecase and weighed/tried the option.

> Postgres search is essentially an easier to use regex engine.

I'm not sure exactly what you meant to convey here, but if you're searching with LIKE or `~` you're not doing Postgres's proper Full Text Search. You should be dealing with tsvectors[0]

> As soon as you need multiple languages

Postgres FTS supports multiple languages and you can create your own configurations[1]

> advanced autocomplete

I'm not sure what "advanced" autocomplete is but you can get pretty fast trigram searches going[2] (back to LIKE/ILIKE here but obviously this is an isolated usecase). In the end I'd expect auto complete results to actually not hit your DB most of the time (maybe I'm naive but that feels like a caching > cache invalidation > cache pushdown problem to me)

> misspelling detection

pg_similarity_extension[3] might be of some help here, but it may require some wrangling.

> large documents, large datasets,

PG has TOAST[4], and obviously can scale (maybe not necessarily great at it) -- see pg_partman/Timescale/Citus/etc.

> custom scoring

Postgres only has basic ranking features[5], but you can write your own functions and extend it of course.

Solr/ES are definitely the right tools for the job (tm) when the job is search, but you can get surprisingly far with Postgres. I'd argue that many usecases actually don't want/need a perfect full text search solution -- it's often minor features that turn into overkill fests and ops people learning/figuring out how to properly manage and scale an ES cluster and falling into pitfalls along the way.

[0]: https://www.postgresql.org/docs/current/textsearch-intro.htm...

[1]: https://www.postgresql.org/docs/current/textsearch-intro.htm...

[2]: https://about.gitlab.com/blog/2016/03/18/fast-search-using-p...

[3]: https://github.com/eulerto/pg_similarity

[4]: https://www.postgresql.org/docs/current/storage-toast.html

[5]: https://www.postgresql.org/docs/9.5/textsearch-controls.html...