dock VS tantivy

Just some friendly feedback: from the page you've linked to and this post, it's a little confusing to figure out what this is and why I should use it over the native Docker CLI. The GitHub page is a little more informative - even adding a framing headline like "A tool for using Docker containers as environments" or "Dock is a thin wrapper around the Docker CLI that makes Docker environments seamless" above the screenshot on your landing page would probably go a long way to communicating what your tool does and who should care.

I'm adding tests around the newest subcommand in dock, which is a thin wrapper around docker to streamline the use of Docker in a "development" context (as opposed to a deployment context). I recently added the run subcommand for running a command in the newest build of a Docker container, which will have certain configurable shortcuts, like running the command with the local user's ID, bind mounting the local project directory, and "nesting" the Docker daemon.

Hi /u/benzaita, thanks for sharing. I'm working on a tool with similar functionality (dock), so it's interesting to see a different approach in the same space. The features and Why use dockerized are great breakdowns of the advantages of this approach, and points that I've often made or observed myself, so it's great to have lists of these points to reference.

Tantivy - a full-text indexing library written in Rust. Has a great performance and featureset.

By this I presume you mean build a search index that can retrieve results based on keywords? I know certain databases use Lucene to build a keyword-based index on top of unstructured blobs of data. Another alternative is to use Tantivy (https://github.com/quickwit-oss/tantivy), a Rust version of Lucene, if building search indices via Java isn't your cup of tea :)

Both libraries offer multilingual support for keywords, I believe, so that's a benefit to vector search where multilingual embedding models are rather expensive.

We also implemented our schemaless columnar storage optimized for object storage.

The inverted index and columnar storage are part of tantivy [0], which is the fastest search library out there. We maintain it and we decided to build the distributed engine on top of it.

[0] tantivy github repo: https://github.com/quickwit-oss/tantivy

The issue for geo search is here: https://github.com/quickwit-oss/tantivy/issues/44

Search index : Custom-built using tantivy.

The roaring bitmap variant is used only for the optional index (1 docid => 0 or 1 value) in the columnar storage (DocValues), not for the inverted index. Since this is used for aggregation, some queries may be a full scan.

The inverted index in tantivy uses bitpacked values of 128 elements with a skip index on top.

> I didn't follow the rest of your comment, select is what EF is good at, every other data structure needs a lot more scanning once you land on the right chunk. With BMI2 you can also use the PDEP instruction to accelerate the final select on a 64-bit block

The select for the sparse codec is a [simple array index access](https://github.com/quickwit-oss/tantivy/blob/main/columnar/s...), that is hard to beat. Compression is not good near the 5k threshold though.

Hi /r/rust, I’m a SWE on Etsy’s Retrieval Systems team where we’re building a platform based on rust and tantivy (https://github.com/quickwit-oss/tantivy). We’re looking to bring two new engineers onto the team.

Quick Search: Easily find specific notes with Velo's fuzzy-search feature, powered by tantivy. tantivy might have been a little overkill, but it was really easy to integrate.