Searchkick VS jq

I run a large scale production application that does something along these lines. If the data needs to be close to real-time, I'd say use `searchkick` + Elasticsearch, and use `searchkick`'s async feature to "stream" the data from your table to the ES index. Your dashboard will then just query from the ES index via searchkick.

You're right, that's actually what we implemented, application-level hooks, but they needed development and maintenance effort that come for free with the adapter we're using for OpenSearch integration, which also comes with welcome features: synonyms, partial matches, and many others.

Spoiler, the adapter is Searchkick: https://github.com/ankane/searchkick

Searchkick

Normally for Rails applications you would use a gem like searchkick since it greatly reduces the initial Elasticsearch complexity.

We lean heavily on Elasticsearch at CompanyCam. One of it's primary use cases is serving our highly filterable project feed. It is incredibly fast, even when you apply multiple filters to your query and are searching a largish data set. Our primary interface for interacting with Elasticsearch is using the Searchkick gem. Searchkick is a powerhouse and provides so many features out of the box. One place where we bump up against the edges is when trying to reindex a large collection.

Convinced? Ok read on and I’ll show you what switching from Elasticsearch to Meilisearch looked like for a real production app — ScribeHub. We also moved from Ankane’s excellent Searchkick gem to the first party meilisearch-rails gem and I’ll show you the changes there as well.

If you want to use a look-aside pattern (like you might have used with Searchkick + Elasticsearch), you should check out ActiveStash: https://github.com/cipherstash/activestash

Searchkick learns what your users are looking for. As more people search, it gets smarter and the results get better. It’s friendly for developers - and magical for your users. BONUS: it's written and supported by "ankane" who has flawless reputation amongst the Ruby community.

That should recursively list directories, counting only the files within each, and output² jsonl that can be further mangled within the shell². You could just as easily populate an associative array for further work, or $whatever. Unlike bash, zsh has reasonable behaviour around quoting and whitespace too.

¹ https://zsh.sourceforge.io/Doc/Release/User-Contributions.ht...

² https://github.com/jpmens/jo

³ https://github.com/stedolan/jq

What I miss from jq and what is implemented but unreleased is platform independent line delimiters.

jq on Windows produces \r\n terminated lines which can be annoying when used with Cygwin / MSYS2 / WSL. The '--binary' option to not convert line delimiters is one of those pending improvements.

https://github.com/stedolan/jq/commit/0dab2b18d73e561f511801...

If you have jq installed you can use it to make the output look nicer.

It requires jq for JSON processing and GNU parallel for concurrent searches in the notebooks.

jq is your friend.

The jq command (https://stedolan.github.io/jq/) is useful pulling that information out.