SimSIMD VS usearch

For other languages (including nodejs/bun/rust/python etc) you can have a look at SimSIMD which I have contributed to this year (made recompiled binaries for nodejs/bun part of the build process for x86_64 and arm64 on Mac and Linux, x86 and x86_64 on windows).

[0] https://github.com/ashvardanian/SimSIMD

Kahan floats are also commonly used in such cases, but I believe there is room for improvement without hitting those extremes. First of all, we should tune the epsilon here: https://github.com/ashvardanian/SimSIMD/blob/f8ff727dcddcd14...

As for the 64-bit version, its harder, as the higher-precision `rsqrt` approximations are only available with "AVX512ER". I'm not sure which CPUs support that, but its not available on Sapphire Rapids.

I encourage one to merge into e.g. {NumPy, SciPy, }; are there PRs?

Though SymPy.physics only yet supports X,Y,Z vectors and doesn't mention e.g. "jaccard"?, FWIW: https://docs.sympy.org/latest/modules/physics/vector/vectors... https://docs.sympy.org/latest/modules/physics/vector/fields.... #cfd

include/simsimd/simsimd.h: https://github.com/ashvardanian/SimSIMD/blob/main/include/si...

conda-forge maintainer docs > Switching BLAS implementation:

Hello, everybody! I was working on the next major release of USearch, and in the process, I decided to generalize its underlying library - SimSIMD. It does one very simple job but does it well - computing distances and similarities between high-dimensional embeddings standard in modern AI workloads.

Hey everyone! I am excited to share updates on four of my & my teams' open-source projects that take large-scale search systems to the next level: USearch, UForm, UCall, and StringZilla. These projects are designed to work seamlessly together, end-to-end—covering everything from indexing and AI to storage and networking. And yeah, they're optimized for x86 AVX2/512 and Arm NEON/SVE hardware.

USearch [1]: Think of it as Meta FAISS on steroids. It's now quicker, supports clustering of any granularity, and offers multi-index lookups. Plus, it's got more native bindings than probably all other vector search engines combined: C++, C, Python, Java, JavaScript, Rust, Obj-C, Swift, C#, GoLang, and even slightly outdated bindings for Wolfram. Need to refresh that last one!

UForm v2 [2]: Imagine a much smaller OpenAI CLIP but more efficient and trained on balanced multilingual datasets, with equal exposure to languages from English, Chinese, and Hindi to Arabic, Hebrew, and Armenian. UForm now supports 21 languages, is so tiny that you can run it in the browser, and outputs small 256-dimensional embeddings. Perfect for rapid image and video searches. It's already available on Hugging-Face as "unum-cloud/uform-vl-multilingual-v2".

UCall [3]: It started as a FastAPI alternative focusing on JSON-RPC (instead of REST protocols), offering 70x the bandwidth and 1/50th the latency. It was good but not enough, so we've added REST and TLS support, broadening its appeal. I've merged that code, and it is yet to be tested. Early benchmarks suggest that we still hit the same 150'000-250'000 requests/s on a single CPU core in Python by reusing HTTPS connections.

StringZilla [4]: This project lets you sift through multi-gigabyte or terabyte strings with minimal use of RAM and maximal use of SIMD and SWAR techniques.

All these projects are engineered for scalability and efficiency, even on tight budgets. Our demo, for instance, works on hundreds of gigabytes of images using just a few gigabytes of RAM and no GPUs for AI inference. That is a toy example with a small, noisy dataset, and I look forward to showing a much larger setup. Interestingly, even this tiny setup illustrates issues common to UForm and much larger OpenAI CLIP models - the quality of Multi-Modal alignment [5]. It also shows how different/accurate the search results are across different languages. Synthetic benchmarks suggest massive improvements for some low-resource languages (like Armenian and Hebrew) and more popular ones (like Hindi and Arabic) [6]. Still, when we look at visual demos like this, I can see a long road ahead for us and the broader industry, making LLMs Multi-Modal in 2024 :)

All of the projects and the demo code are available under an Apache license, so feel free to use them in your commercial projects :)

PS: The demo looks much nicer with just Unsplash dataset of 25'000 images, but it's less representative of modern AI datasets, too small, and may not be the best way to honestly show our current weaknesses. The second dataset - Conceptual Captions - is much noisier, and quite ugly.

[1]: https://github.com/unum-cloud/usearch

Another worth mentioning in this thread is usearch, though not a separate algorithm, based on HNSW with a bunch of optimizations https://github.com/unum-cloud/usearch

I've used usearch successfully for a small project: https://github.com/unum-cloud/usearch/

The hardest (still missing) part of efficient cosine computation distance computation is picking a good epsilon for the `sqrt` calculation and avoiding "division by zero" problems.

We have an open issue about it in USearch and a related one in SimSIMD itself, so if you have any suggestions, please share your insights - they would impact millions of devices using the library (directly on servers and mobile, and through projects like ClickHouse and some of the Google repos): https://github.com/unum-cloud/usearch/issues/320

As you scale, you may benefit from these two projects I maintain, and the Big Tech uses :)

https://github.com/unum-cloud/usearch - for faster search

https://github.com/unum-cloud/uform - for cheaper multi-lingual multi-modal embeddings

> It doesn't appear to query CPUID

Yes, I'm actually looking for a good way to do it for other projects as well. I've looked into a couple more libs, and here is the best I've come up with so far: https://github.com/unum-cloud/usearch/blob/f942b6f334b31716f...

> Your substring routines have multiplicative worst case

Yes, that is true. It's a very simple stupid trick, just happens to work well for me :)

> It seems quite likely that your confirmation step

We have a different library internally at Unum, that avoids this shortcoming. It has a few thousand lines of C++ templates with SIMD intrinsics... and it's definitely more efficient, but the margins aren't always high. So I kept the pure C version with inlined functions as minimal and simple as possible.

> It would actually be possible to hook Stringzilla up to `memchr`'s benchmark suite if you were interested. :-)

Yes, that would be a fun thing to do! I haven't had time to look into `memchr` yet, but would expect great perf from your lib as well. For me the State of the Art is Intel HyperScan. Probably the most advanced SIMD library overall, not just for strings. I was very impressed with their perf ~5 years ago. But the repo is 200 K LOC... So get ready to invest a weekend :)

That said, I'm a bit slammed with work right now, including open-source. Hoping to ship a new major release in UCall this week, and a minor one in USearch :)

We don't use BLAS. Why? BLAS helps with matrix-matrix multiplications, if you feel lazy and don't want to write the matrix tiling code manually.

They bring essentially nothing of value in vector-vector operations, as compilers can properly auto-vectorize simple dot products... Moreover, they generally only target single and double precision, while we often prefer half or quarter precision. All in all, meaningless dependency.

What do we use? I wrote a tiny package called SimSIMD. It's idea is to utilize less common SIMD instructions, especially in mixed-typed computations, that are hard for compilers to optimize. It was also a fun exercise to evaluate the performance of new SVE instruction on recent Arm CPUs, like the Graviton 3. You can find the code, the benchmarks, and the results in the repo: https://github.com/ashvardanian/simsimd

Still, even without SimSIMD, USearch seems to be one of the faster implementations of vector search. You can find the benchmarks in the first table here: https://github.com/unum-cloud/usearch#memory-efficiency-down...