veneur
opentelemetry-lambda
veneur | opentelemetry-lambda | |
---|---|---|
2 | 8 | |
1,714 | 243 | |
0.1% | 0.8% | |
3.5 | 9.3 | |
about 1 month ago | 6 days ago | |
Go | Go | |
MIT License | Apache License 2.0 |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
veneur
-
OpenTelemetry in 2023
This was the idea behind Stripe's Veneur project - spans, logs, and metrics all in the same format, "automatically" rolling up cardinality as needed - which I thought was cool but also that it would be very hard to get non-SRE developers on board with when I saw a talk about it a few years ago.
https://github.com/stripe/veneur
-
Launch HN: Opstrace (YC S19) – open-source Datadog
One pain point with Prometheus is that is has relatively weak support for quantiles, histograms, and sets[1]:
- Histograms require manually specifying the distribution of your data, which is time-consuming, lossy, and can introduce significant error bands around your quantile estimates.
- Quantiles calculated via the Prometheus "summary" feature are specific to a given host, and not aggregatable, which is almost never what you want (you normally want to see e.g. the 95th percentile value of request latency for all servers of a given type, or all servers within a region). Quantiles can be calculated from histograms instead, but that requires a well-specified histogram and can be expensive at query time.
- As far as I know, Prometheus doesn't have any explicit support for unique sets. You can compute this at query time, but persisting and then querying high-cardinality data in this way is expensive.
Understanding the distribution of your data (rather than just averages) is arguably the most important feature you want from a monitoring dashboard, so the weak support for quantiles is very limiting.
Veneur[2] addresses these use-cases for applications that use DogStatsD[3] by using clever data structures for approximate histograms[4] and approximate sets[5], but I believe its integration with Prometheus is limited and currently only one-way - there is a CLI app to poll Prometheus metrics and push them into Veneur, but there's no output sink for Veneur to write to Prometheus (or expose metrics for a Prometheus instance to poll).
It would be extremely useful to have something similar for Prometheus, either by integrating with Veneur or implementing those data structures as an extension to Prometheus.
[1] https://prometheus.io/docs/practices/histograms/
[2] https://github.com/stripe/veneur
[3] https://docs.datadoghq.com/developers/dogstatsd/
[4] https://github.com/stripe/veneur#approximate-histograms
[5] https://github.com/stripe/veneur#approximate-sets
opentelemetry-lambda
-
Did OpenTelemetry deliver on its promise in 2023?
I mean, sure, you can improve performance a bit by increasing the RAM/compute capacity on the Lambda. But it always adds a pretty steep overhead right now, no matter how much capacity you throw at it.
https://github.com/open-telemetry/opentelemetry-lambda/issue...
https://github.com/aws-observability/aws-otel-lambda/issues/...
-
Instrumenting AWS Lambda functions with OpenTelemetry SDKs
OpenTelemetry AWS Lambda repository
- OpenTelemetry in 2023
-
Serverless Spy Vs. Spy Chapter 3: X-Ray vs Jaeger - Send Lambda traces with open telemetry
With the sample apps from the opentelemetry-lambda repository the Lambda part itself was easy to implement. What took me some time was to provide the jaeger Fargate service with IaC ouside of an k8s environment. But with ECS and ServiceDiscovery that was easy in the end. This should be even more simple in an EKS environment with the jaegertracing helm-charts.
-
AWS Lambda tracing with OpenTelemetry and OpenSearch
OpenTelemetry recently released https://github.com/open-telemetry/opentelemetry-lambda, but they also have this in the official docs https://opentelemetry.io/docs/instrumentation/js/serverless/. What do you consider to be the better option?
-
Serverless Spy Vs. Spy Chapter 2: AWS Distro for OpenTelemetry Lambda vs X-Ray SDK
opentelemetry-lambda
-
How to Instrument AWS Services with OpenTelemetry
You don’t have to create an opentelemetry configuration file such as this for each of your lambdas. In fact, you shouldn’t. In AWS, you can use Lambda Layers. You can define the OpenTelemetry tracing piece of code as a Lambda layer and use it in any Lambda you want. Furthermore, OpenTelemetry went ahead and implemented this opentelemetry-lambda layer for us. All we need to do is use it with our config.
-
Struggling to connect the dots - ADOT with Lambda using aws-otel-nodejs Lambda layer, not sure how to go from here to using custom instrumentation (e.g. instrumentation-pg, instrumentation-graphql, etc).
Sorry you're having trouble working with the ADOT Lambda Layers :(. Have you had a chance to open an issue on the GitHub repo for OTel Lambda or ADOT Lambda? You should add your expected vs your actual output!
What are some alternatives?
opstrace - The Open Source Observability Distribution
terraform-aws-lambda - Terraform module, which takes care of a lot of AWS Lambda/serverless tasks (build dependencies, packages, updates, deployments) in countless combinations 🇺🇦
cortex - A horizontally scalable, highly available, multi-tenant, long term Prometheus.
deploy-aws-lambda-to-vpc-with-terraform - Terraform module with all the cloud resources needed to run Lambda within a VPC
Cortex - Cortex: a Powerful Observable Analysis and Active Response Engine
sqs-consumer - Build Amazon Simple Queue Service (SQS) based applications without the boilerplate
loki - Like Prometheus, but for logs.
opentelemetry-examples
influxdb-apply - Define InfluxDB users and databases with a yaml file.
aws-otel-js - AWS Distro for OpenTelemetry JavaScript SDK
b3-propagation - Repository that describes and sometimes implements B3 propagation
helm-charts - Helm Charts for Jaeger backend