Redis Streams là gì? So sánh với Pub/Sub và Kafka, khi nào dùng?

Redis Streams (Redis 5.0) là log data structure lưu message theo thứ tự với persistent storage, consumer group, và message acknowledgment — lấp đầy khoảng cách giữa Pub/Sub (fire-and-forget) và Kafka (phức tạp, separate infrastructure).

• Mỗi message có auto-generated ID dạng timestamp-sequence (ví dụ: 1638000000000-0).
• Consumer group trong Redis Streams tương tự Kafka: XREADGROUP GROUP mygroup consumer1 COUNT 10 STREAMS mystream > đọc undelivered message; XACK mystream mygroup message-id acknowledge; XPENDING kiểm tra message chưa ack (để xử lý lại nếu consumer crash). XADD stream * field1 value1 thêm message; MAXLEN ~1000 để cap stream size.
• Dùng Redis Streams khi: cần reliable messaging nhưng không muốn setup Kafka, data volume vừa phải, team đã dùng Redis.
• Dùng Kafka khi: data volume cực lớn, cần long-term retention, multi-datacenter replication, hoặc cần Kafka ecosystem (Kafka Streams, Connect).

Redis Streams (introduced in Redis 5.0) is an append-only log data structure with persistent storage, consumer groups, and message acknowledgment — bridging the gap between Pub/Sub (fire-and-forget) and Kafka (complex, separate infrastructure).

• Each message gets an auto-generated ID in the format timestamp-sequence (e.g., 1638000000000-0).
• Consumer groups in Redis Streams work similarly to Kafka: XREADGROUP GROUP mygroup consumer1 COUNT 10 STREAMS mystream > reads undelivered messages; XACK mystream mygroup message-id acknowledges; XPENDING checks unacknowledged messages for reprocessing after a consumer crash. XADD stream * field1 value1 appends a message; MAXLEN ~1000 caps the stream size.
• Use Redis Streams when: you need reliable messaging without setting up Kafka, data volume is moderate, and your team already uses Redis.
• Use Kafka when: data volume is very large, long-term retention is needed, multi-datacenter replication is required, or you need the Kafka ecosystem (Kafka Streams, Kafka Connect).