Ask HN: Are there any reliable benchmarks for Machine Learning Model Serving?

• cortex

  33 7,989 2.2 Go

  Production infrastructure for machine learning at scale

  

• llama.cpp

  769 56,891 10.0 C++

  LLM inference in C/C++

Not exactly what you’re looking fir, but oerhaps you’ll find it useful - llama benchmarked on all M-series chips, and in comments there are comparisons with nvidia.

https://github.com/ggerganov/llama.cpp/discussions/4167

• InfluxDB

  www.influxdata.com sponsored

  Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

  

Suggest a related project