Structured logging với correlation ID hoạt động như thế nào trong microservices?

Mục tiêu: request đi qua 5 service, log của cả 5 phải filter được theo 1 key duy nhất.

Cơ chế:
1. MDC (Mapped Diagnostic Context) — thread-local map chứa traceId, spanId, requestId, userId.
2. Mỗi service log JSON với MDC field:

{"traceId":"abc123", "userId":"u1", "level":"INFO", "msg":"Processing order"}

3. OpenTelemetry/Micrometer tự populate MDC từ trace header.
4. Centralized log: ELK Stack hoặc Grafana Loki nhận log từ tất cả service → query traceId = "abc123" để xem toàn bộ flow.

Setup:

<dependency>
  <groupId>net.logstash.logback</groupId>
  <artifactId>logstash-logback-encoder</artifactId>
</dependency>

<!-- logback-spring.xml -->
<appender name="JSON" class="ch.qos.logback.core.ConsoleAppender">
  <encoder class="net.logstash.logback.encoder.LogstashEncoder"/>
</appender>

Lợi ích: MTTR (Mean Time to Resolution) từ hàng giờ xuống phút.

Caveat Virtual Threads (Spring Boot 3.2+): MDC là thread-local — Virtual Threads reuse có thể leak context giữa request. Dùng Micrometer Observation API thay vì raw MDC để propagate đúng.

Goal: a request traverses 5 services, the logs of all 5 must be filterable by a single key.

Mechanism:
1. MDC (Mapped Diagnostic Context) — a thread-local map holding traceId, spanId, requestId, userId.
2. Each service logs JSON with MDC fields:

{"traceId":"abc123", "userId":"u1", "level":"INFO", "msg":"Processing order"}

3. OpenTelemetry/Micrometer auto-populates MDC from trace headers.
4. Centralised logs: ELK Stack or Grafana Loki aggregate logs from every service → query traceId = "abc123" to see the whole flow.

Setup:

<dependency>
  <groupId>net.logstash.logback</groupId>
  <artifactId>logstash-logback-encoder</artifactId>
</dependency>

<!-- logback-spring.xml -->
<appender name="JSON" class="ch.qos.logback.core.ConsoleAppender">
  <encoder class="net.logstash.logback.encoder.LogstashEncoder"/>
</appender>

Benefit: MTTR (Mean Time to Resolution) drops from hours to minutes.

Virtual Threads caveat (Spring Boot 3.2+): MDC is thread-local — reused Virtual Threads can leak context across requests. Use the Micrometer Observation API instead of raw MDC for correct propagation.