-
InfluxDB
Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
We've built it for apps that use LLMs and other ML models. The lightweight Python agent autoinstruments OpenAI, LangChain, Banana, and other APIs and frameworks. Basically by adding one line of code you'll be able to monitor and analyze latency, errors, compute and costs. Profiling using CProfile, PyTorch Kineto or Yappi can be enabled if code-level statistics are necessary.
Here is a short demo screencast for a LangChain/OpenAI app: https://www.loom.com/share/17ba8aff32b74d74b7ba7f5357ed9250
In terms of data privacy, we only send metadata and statistics to https://graphsignal.com. So no raw data, such as prompts or images leave your app.
Related posts
-
Show HN: Python Monitoring for AI: LLMs, OpenAI, Inference, GPUs
-
Show HN: Python Monitoring for AI: LLMs, OpenAI, Inference, GPUs
-
[N] Monitor OpenAI API Latency, Tokens, Rate Limits, and More with Graphsignal
-
Monitor OpenAI API Latency, Tokens, Rate Limits, and More
-
[N] Easily profile FastAPI model serving