fst VS rust

I'd point you towards BurntSushi's fst crate: https://github.com/BurntSushi/fst

The fst crate effectively relies on mmap for it to work right. The folks here suggesting you just use the heap might be right, but only if using the heap is actually plausible. If your dictionary is GBs big (an FST might be bigger than available memory), then copying it the heap first would be disastrous.

You'll love what we're working on if you're interested in the implementation of:- Tantivy- Meilisearch- Finite State Transducers

Here's an example of longest prefix matching using a FST which I based my approach on: https://github.com/BurntSushi/fst/pull/104/files

Finite State Transducers

As a result, I have a plan to implement a desktop version from then and I finished today with a beta version. The desktop is based on tauri, and the dictionary index algorithm is FST (it is an awesome index algorithm).

WordBueno’s data is currently derived from Wiktionary. The backend is using Rust’s warp with fst for indexing.

No you don't. I've written multiple programs that load things instantly off the file system via memory maps. See the fst crate[1], for example, which is designed to work with memory maps.

Rust "works badly with memory mapped files" doesn't mean, "Rust can't use memory mapped files." It means, "it is difficult to reconcile Rust's safety story with memory maps." ripgrep for example uses memory maps because they are faster sometimes, and its safety contract[2] is a bit strained. But it works.

[1] - https://github.com/BurntSushi/fst/

[2] - https://docs.rs/grep-searcher/0.1.7/grep_searcher/struct.Mma...

Almost fixed the compiler: https://github.com/rust-lang/rust/pull/123325

Rust: A secure, efficient, and modern programming language (omitting ten thousand words). You can simply follow the installation instructions provided on the official website.

Recently did something similar in Rust but for generating SVGs. We've adopted it for snapshot testing of cargo and rustc's output. Don't have a good PR handy for showing Github's rendering of changes in the SVG (text, side-by-side, swiping) but https://github.com/rust-lang/rust/pull/121877/files has newly added SVGs.

To see what is supported, see the screenshot in the docs: https://docs.rs/anstyle-svg/latest/anstyle_svg/

We strongly believe in Rust as a powerful language for building production-grade software, especially for systems like ours that run alongside Kubernetes.

The above Assert<{N % 2 == 1}> requires #![feature(generic_const_exprs)] and the nightly toolchain. See https://github.com/rust-lang/rust/issues/76560 for more info.

Another week, another dive into Rust. This time, we're delving into structs. Structs bear resemblance to interfaces in TypeScript, enabling the grouping of intricate data sets within an object, much like TypeScript/JavaScript. Rust also accommodates functions within these structs, offering a semblance of classes, albeit with distinctions. Let's delve into this topic.

There’s also other reasons. For example, take binary search:

* prefetch + cmov. These should be part of the STL but languages and compilers struggle to emit the cmov properly (Rust’s been broken for 6 years: https://github.com/rust-lang/rust/issues/53823). Prefetch is an interesting one because while you do optimize the binary search in a micro benchmark, you’re potentially putting extra pressure on the cache with “garbage” data which means it’s a greedy optimization that might hurt surrounding code. Probably should have separate implementations as binary search isn’t necessarily always in the hot path.

* Eytzinger layout has additional limitations that are often not discussed when pointing out “hey this is faster”. Adding elements is non-trivial since you first have to add + sort (as you would for binary search) and then rebuild a new parallel eytzinger layout from scratch (i.e. you’d have it be an index of pointers rather than the values themselves which adds memory overhead + indirection for the comparisons). You can’t find the “insertion” position for non-existent elements which means it can’t be used for std::lower_bound (i.e. if the element doesn’t exist, you just get None back instead of Err(position where it can be slotted in to maintain order).

Basically, optimizations can sometimes rely on changing the problem domain so that you can trade off features of the algorithm against the runtime. These kinds of algorithms can be a bad fit for a standard library which aims to be a toolbox of “good enough” algorithms and data structures for problems that appear very very frequently. Or they could be part of the standard library toolkit just under a different name but you also have to balance that against maintenance concerns.