-
To add telemetry to our app you will use OpenTelemetry, an open-source observability framework for generating, capturing, and collecting telemetry data such as logs, metrics, and traces from software services and applications.
-
CodeRabbit
CodeRabbit: AI Code Reviews for Developers. Revolutionize your code reviews with AI. CodeRabbit offers PR summaries, code walkthroughs, 1-click suggestions, and AST-based analysis. Boost productivity and code quality across all major languages with each PR.
-
tracetest
🔠Tracetest - Build integration and end-to-end tests in minutes, instead of days, using OpenTelemetry and trace-based testing.
For this article, I’ll show a simplified version of the code, to show how we can trigger an LLM task, expose it via API and then instrument it. You can see the complete source code, here.
-
To instrument our app you will use the CLI tool opentelemetry-instrument that automatically sets up auto-instrumentation in your code without needing to do boilerplate configuration, and OTel Python SDK and OpenLLMetry to do manual instrumentation and specific instrumentation for LLM SDKs, like the file ./app/telemetry.py in the example:
-
This class of AI algorithms has been widely used in enterprise-level applications due to its capability of contextual understanding, being scalable, and handling large volumes of text data, among other features. With simple integration APIs like OpenAI, Google Gemini, Hugging Face, and Antrophic, and good frameworks that deal with external providers and in-house models, like LangChain, developers can implement interesting applications that use LLMs for internal tasks.
-
By running the make start/on-docker command at beginning of the section, you already started the API and also an Observability stack with an OpenTelemetry Collector and Jaeger.
-
This test does a call to our API and performs some simple assertions to check if you received a proper output, a valid JSON with a summary field in it. You also could do some tests to see if the text is relevant and have low accuracies (like the Python deepeval lib does).
Related posts
-
Amazon EKS Monitoring with OpenTelemetry [Step By Step Guide]
-
Ending Microservices Chaos: How Architecture Governance Keeps Your Microservices on Track
-
Desbloqueie a Observabilidade: Guia prático com OpenTelemetry e Prometheus
-
Prometheus vs CloudWatch for Cloud Native Applications (Updated in 2024)
-
OpenTelemetry and vendor neutrality: how to build an observability strategy