Flaky tests là gì? Nguyên nhân và cách xử lý?

Flaky tests là tests đôi khi pass, đôi khi fail với cùng code — cực kỳ độc hại cho CI/CD vì giảm trust vào test suite và làm team bỏ qua failing tests.

Nguyên nhân phổ biến:

1. Timing/async issues — test không chờ đúng async operation, dùng arbitrary setTimeout thay vì waitFor/explicit await;
2. Test order dependency — test A phụ thuộc vào side effects của test B;
3. Shared mutable state — global state không reset giữa tests;
4. External dependencies — gọi real APIs, database không isolated;
5. Date/time — dùng new Date() thay vì mock;
6. Random data — không seed random generator

Cách xử lý: waitFor(() => expect(element).toBeInTheDocument()) trong Testing Library thay vì setTimeout; beforeEach reset state; mock Date.now() với jest.useFakeTimers(); server.resetHandlers() giữa tests.

Phát hiện: chạy test file nhiều lần jest --testPathPattern=flaky.test.ts --runInBand --repeat 50; Playwright có --repeat-each.

CI retry logic (GitHub Actions retry): cảnh báo nếu test cần retry > 1 lần — không xem là solution, chỉ là detection mechanism.

Flaky tests sometimes pass and sometimes fail with the same code — extremely harmful for CI/CD because they erode trust in the test suite and cause teams to ignore failures.

Common causes:

1. Timing/async issues — test doesn't wait for async operations correctly, uses arbitrary setTimeout instead of waitFor/explicit await;
2. Test order dependency — test A relies on side effects from test B;
3. Shared mutable state — global state not reset between tests;
4. External dependencies — calling real APIs, non-isolated database;
5. Date/time — using new Date() instead of mocking;
6. Random data — unseeded random generator

Fixes: waitFor(() => expect(element).toBeInTheDocument()) in Testing Library instead of setTimeout; beforeEach state reset; mock Date.now() with jest.useFakeTimers(); server.resetHandlers() between tests.

Detection: run the test file many times — jest --testPathPattern=flaky.test.ts --runInBand --repeat 50; Playwright has --repeat-each.

CI retry logic is a detection mechanism, not a solution — a test that requires retry is still flaky and needs to be fixed.