⚡ Distributed Metrics System

Spring Boot Kafka Topic Apache Flink Apache Spark Database MinIO HDFS Click any node → details  |  Drag to rearrange
publish publish consume enrich lookup normalized Debezium CDC P2 P1 broadcast JDBC upsert Parquet FileSink rolling txt batch read write Parquet Structured Data JSON / Avro / Protobuf Log / Text Data raw lines, binary metrics-collector Spring Boot · :8081 metrics.structured Kafka topic logs.raw Kafka topic metrics.normalized Kafka topic cdc.public .service_metadata metrics-processor Spring Boot · :8082 Flink Job Pipeline 1 — windowed aggregation Pipeline 2 — log archival to HDFS Pipeline 3 — user events (Protobuf) PostgreSQL enrichment + CDC · :5432 TimescaleDB time-series · :5433 HDFS raw .txt log files MinIO Parquet · S3-compat · :9000 API Gateway Spring Boot · :8083 Spark Job batch · spark-submit HDFS Parquet structured log records Protobuf P3 register schema cache JSON ingest P4 NDJSON files Parquet S3 (opt.) users.events Kafka · Protobuf Schema Registry Confluent · :8081 external.data.raw Kafka · raw JSON HDFS external NDJSON ExternalData Processor Spark · schema-on-read

👆 Click any node to
learn what it does,
what talks to it, and
see real code examples.

✋ Drag nodes to
rearrange the diagram.

Follow a data path: