Searchkick VS lockbox

I run a large scale production application that does something along these lines. If the data needs to be close to real-time, I'd say use `searchkick` + Elasticsearch, and use `searchkick`'s async feature to "stream" the data from your table to the ES index. Your dashboard will then just query from the ES index via searchkick.

You're right, that's actually what we implemented, application-level hooks, but they needed development and maintenance effort that come for free with the adapter we're using for OpenSearch integration, which also comes with welcome features: synonyms, partial matches, and many others.

Spoiler, the adapter is Searchkick: https://github.com/ankane/searchkick

Searchkick

Normally for Rails applications you would use a gem like searchkick since it greatly reduces the initial Elasticsearch complexity.

We lean heavily on Elasticsearch at CompanyCam. One of it's primary use cases is serving our highly filterable project feed. It is incredibly fast, even when you apply multiple filters to your query and are searching a largish data set. Our primary interface for interacting with Elasticsearch is using the Searchkick gem. Searchkick is a powerhouse and provides so many features out of the box. One place where we bump up against the edges is when trying to reindex a large collection.

Convinced? Ok read on and I’ll show you what switching from Elasticsearch to Meilisearch looked like for a real production app — ScribeHub. We also moved from Ankane’s excellent Searchkick gem to the first party meilisearch-rails gem and I’ll show you the changes there as well.

If you want to use a look-aside pattern (like you might have used with Searchkick + Elasticsearch), you should check out ActiveStash: https://github.com/cipherstash/activestash

Searchkick learns what your users are looking for. As more people search, it gets smarter and the results get better. It’s friendly for developers - and magical for your users. BONUS: it's written and supported by "ankane" who has flawless reputation amongst the Ruby community.

The first step is to install the gem with bundle add authtrail. Additionally, since you'll be storing user-identifiable information such as emails and IP addresses in your app database, it's highly recommended that you encrypt this data in production using a combination of Lockbox and Blindindex gems.

Your app queries an encrypted ActiveStash index, which says what rows in your existing database should be retrieved. It works with all the popular application-level encryption plugins for ActiveRecord, including ActiveRecord Encryption, and Lockbox.

Content is encrypted using https://github.com/ankane/lockbox default setup I've been considering adding optional end-to-end encryption as well, but am a bit afraid people might lose access to content forever if they lose access to keys.

lockbox and blind_index for email fields encryption

Install lockbox and blind_index.

https://github.com/ankane/lockbox this is for the actual encryption of data

Lockbox

I've been using the lockbox gem to encrypt specified attributes.

I stumbled upon lockbox when looking for alternatives to attr_encrypted (It hadn't been updated for a while). It's really super & I'm pretty keen to share it :D