infino VS search-benchmark-game

An example notebook that shows how to use LangChain is here, we have basically added a new callback in LangChain that stores prompt input, response, latency, errors, and token usage in Infino - https://github.com/infinohq/infino/blob/main/examples/llm-monitoring-langchain/llm-monitoring-langchain.ipynb

Infino is a fast and scalable service to store time series and logs - written in Rust. It now integrates with LangChain, the most popular framework to build applications using the LLMs.

Metrics are ingested using a REST api which can be called from existing metrics/logs processors such as logstash, filebit, fluentd and fluentbit. We currently have only an example of fluentbit though, see here - https://github.com/infinohq/infino/tree/main/examples/fluentbit

Also, we have a benchmark for search. Feel free to add your engine. I believe it is fair: we are not leading the leaderboard, the rules are fairly clear, and no one has contested them so far. https://github.com/quickwit-oss/search-benchmark-game/

Could you update the benchmark? It still uses tantivity 0.16.

This is very very difficult, but Tantivy tried: see https://github.com/quickwit-oss/search-benchmark-game

That's just because there's no a lucene equivalent C library with the same level of attention?

however, there are increasingly such written in C++ (pisa) and rust (tantivy). They handily beat lucene in benchmark suites [1] - so it seems like lucene does suffer from a java penalty - despite getting even more developer attention than pisa and tantivy I would think.

1: https://tantivy-search.github.io/bench/

The benchmark is open sourced here: https://github.com/tantivy-search/search-benchmark-game