jmx_exporter
kafka-manager
Our great sponsors
jmx_exporter | kafka-manager | |
---|---|---|
8 | 13 | |
2,896 | 11,670 | |
1.5% | 0.4% | |
8.0 | 0.0 | |
7 days ago | 9 months ago | |
Java | Scala | |
Apache License 2.0 | Apache License 2.0 |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
jmx_exporter
-
Prometheus JMX Exporter for Java17
not quite sure if this is where I should be asking for help but i'm kind of befuddled about this whole situation. I'm trying to set up https://github.com/prometheus/jmx_exporter for our containerized Java application on our cluster. Specifically the JavaAgent as we are interested in getting the CPU and memory metrics especially. However, getting it initialized I am faced with this:
-
Initial setup
Prometheus doesn't deal with logs at all - only metrics. Even just for metrics, you have several different choices depending on how your Java application is running. You may be able to use the JMX Exporter to expose the metrics that you would then scrape with Prometheus. For the logs, you'll need something like Fluent Bit as an agent to grab ('tail' in Fluent Bit input terms) the logs and send them to Elasticsearch or Grafana Loki.
- Pod log monitoring
- Monitor Java application with prometheus
-
What kind of monitoring tools are people using for their Kafka Deployment?
- https://github.com/prometheus/jmx_exporter - https://github.com/lightbend/kafka-lag-exporter (Export Kafka Topics Lag Metrics) - https://github.com/danielqsj/kafka_exporter (Export Kafka Metrics) - https://github.com/obsidiandynamics/kafdrop (Topic Administration tool - create/edit/delete topics & view messages)
-
Kafka monitoring
Usually I use a javaagent that transforms the JMX metrics into prometheus style metrics, for example: https://github.com/prometheus/jmx_exporter/blob/116a1c4b3fafcfbb9e566b34e130ccc805f331fc/example_configs/kafka-0-8-2.yml
-
Do you use Tomcat ? How to you get optimal performance of it?
Another good one is to add the jmx_exporter agent plugin. This allows easy collection of real-time performance metrics.
-
Kafka Connect: The Magic Behind Mux Data Realtime Exports
In our case, we add sink connectors for AWS Kinesis and Google Cloud PubSub. We also add a Prometheus exporter JAR that scrapes the Kafka Connect JMX metrics and exposes them as Prometheus metrics.
kafka-manager
- FLaNK Stack Weekly 16 October 2023
-
UI for Apache Kafka - An open-source tool for monitoring and managing Apache Kafka Clusters - v0.17 release
Are there any comparison to CMAK or Kafdrop ?
-
Running multi-broker Kafka using docker
Dockerized kafka manager (Yahoo CMAK)
- UI for Serverless AWS MSK (Kafka)
-
A list of GUI tools for working with Apache Kafka
Cluster Manager for Apache Kafka (CMAK)
-
What companies/startups are using Scala (open source projects on github)?
There are so many of them in big data, e.g. Kafka, Spark, Flink, Delta, Snowplow, Finagle, Deequ, CMAK, OpenWhisk, Snowflake, TheHive, TVM-VTA, etc.
-
Is there recommed UI for Kafka like RabbitMQ?
We're using CMAK (previously known as kafka manager) https://github.com/yahoo/CMAK
-
Any advice on setting up or working with Kafka?
Some random tips: * Use SSDs, not magnetic disks. Our Kafka brokers used to use magnetic disks for more throughput, but this caused wayyyy more problems, like very slow broker restarts. * You'll want to install something like Burrow so you can get better lag metrics. * You might want to install CMAK. It's a web interface for common ops tasks.
- What kind of monitoring tools are people using for their Kafka Deployment?
-
Gathering opinions on kafka management tools
I've used Yahoo's CMAK before for these things -
What are some alternatives?
Telegraf - The plugin-driven server agent for collecting & reporting metrics.
akhq - Kafka GUI for Apache Kafka to manage topics, topics data, consumers group, schema registry, connect and more...
kafka-lag-exporter - Monitor Kafka Consumer Group Latency with Kafka Lag Exporter
kowl - Redpanda Console is a developer-friendly UI for managing your Kafka/Redpanda workloads. Console gives you a simple, interactive approach for gaining visibility into your topics, masking data, managing consumer groups, and exploring real-time data with time-travel debugging. [Moved to: https://github.com/redpanda-data/console]
kafka_exporter - Kafka exporter for Prometheus
kafka-ui - Open-Source Web UI for Apache Kafka Management
container-jfr - Secure JDK Flight Recorder management for containerized JVMs
kafdrop - Kafka Web UI
exporterhub.io - A Curated List of Prometheus Exporters
Burrow - Kafka Consumer Lag Checking