kafka-lag-exporter

Kafka Lag Exporter - consumer lag monitoring demo.

This sample demonstrates how to monitor consumer lag using Kafka Lag Exporter and visualize metrics in Grafana dashboards.

Grafana dashboard used in this demo can be found here

Components

• Kafka Lag Exporter
• Prometheus
• Grafana

Dashboards

Getting Started

Prerequisites

• Java 11
• Docker

Build

• Build sample Kafka application using below command:

  ./gradlew bootBuildImage

Usage

• Run the docker compose stack.

  cd ./docker/kafka-lag-exporter
  docker compose up -d

• Check if all components are running.

  docker compose ps
  
  # NAME                   SERVICE                STATUS              PORTS
  # grafana                grafana                running (healthy)   0.0.0.0:3000->3000/tcp, :::3000->3000/tcp
  # kafka                  kafka                  running             0.0.0.0:9092->9092/tcp, :::9092->9092/tcp, 0.0.0.0:9101->9101/tcp, :::9101->9101/tcp
  # kafka-lag-exporter     kafka-lag-exporter     running             0.0.0.0:8000->8000/tcp, :::8000->8000/tcp
  # kafka-monitoring-app   kafka-monitoring-app   running             0.0.0.0:7777->7777/tcp, :::7777->7777/tcp
  # prometheus             prometheus             running (healthy)   0.0.0.0:9090->9090/tcp, :::9090->9090/tcp
  # zookeeper              zookeeper              running             0.0.0.0:2181->2181/tcp, :::2181->2181/tcp, 2888/tcp, 3888/tcp

• Open your web browser and go to Grafana login page.

  • Username: admin
  • Password: password

• Open Kafka Lag Exporter dashboard and investigate exposed metrics.

• Stop docker compose stack.

  docker compose down -v

Important Endpoints

Name	Endpoint
Kafka Lag Exporter - Metric endpoint	http://localhost:8000/
Grafana	http://localhost:3000/
Prometheus	http://localhost:9090/

References

• Lightbend Kafka Lag Exporter
• Kafka Lag Exporter Grafana Dashboard
• Prometheus
• Grafana