opentelemetry-rust VS rust-rdkafka

I believe if you set up an Open Telemetry tracer, you can use the Datadog Lambda Extension to send those traces (as well as logs and some additional lambda metrics) to Datadog.

A lot of APM products these days support OpenTelemetry in addition to their custom APM agents. You could try using https://github.com/open-telemetry/opentelemetry-rust paired with an APM provider that works with OpenTelemetry.

(Source: https://github.com/open-telemetry/opentelemetry-rust/blob/main/examples/http/src/client.rs)

Here’s a recent example: https://github.com/open-telemetry/opentelemetry-rust/pull/881

dd-trace-layer is a middleware for sending Datadog's trace. It's based on Tower and OpenTelemetry Rust.

I plan to use OpenTelemetry for my async project, but I'm a bit unclear about whether to use the `tracing`, native `opentelemetry` or even the `tracing-opentelemetry` crate? Both tracing and opentelemetry support instrumenting async/await code. What is the relationship between tracing and opentelemetry? If Otel is trying to standardize a spec for observability across languages and vendors and supports async/await rust code, what's the motivation to learn the `tracing` crate's API at this time?

The git repository called opentelemetry-rust contains implementations of several crates that expand the opentelemetry ecosystem. There you can find common instrumentations, exporters, and subscribers use them in a Rust program.

Not sure about the Aspecto exporter you are using but the last time I tried to use the Jaeger one it seemed fundamentally broken if you had even a moderate amount of spans in a way that keeps popping up and the developers keep closing with various suggestions for workarounds (on the issues of previous people encountering it) that did not work for me ( https://github.com/open-telemetry/opentelemetry-rust/issues/851 ).

You could give open telemetry a try. I've been meaning to check it out, but haven't had a chance yet. https://github.com/open-telemetry/opentelemetry-rust

Would love to see this work with OpenTelemetry: https://github.com/open-telemetry/opentelemetry-rust

I use this library a lot and it's got some nice touches for how to handle wrapping a C library: https://github.com/fede1024/rust-rdkafka

For this example, we're using rdkafka to build producers and consumers, because it allows us to specify custom headers for each record.

kafka

You might find https://crates.io/crates/rdkafka helpful

I am playing with tokio and rust-rdkafka library, following the examples like this one: https://github.com/fede1024/rust-rdkafka/blob/6fb2c37/examples/asynchronous_processing.rs

The source for the current version of the library can be found on Github. I had to increase the major version because I needed to break the API in order to support all formats supported by the current Schema Registry version. I also added the possibility to set an API key, so it can be used with Confluent Cloud, the cloud offering from Confluent. As part of the latest major refactoring it's also supporting async. This might improve performance of your app, and is also the default for the major Kafka client, more information about why you would want to use async can be found in the async book. The schemas retrieved from the Schema Registry are cached. This way the schema is only retrieved once for each id, and reused for other messages with the same id.

What's wrong with rust-rdkafka?

This is a two-part series to help you get started with Rust and Kafka. We will be using the rust-rdkafka crate which itself is based on librdkafka (C library).