Root cause analysis: AI that summarises failure analyses

Forty tests fail after a release. Usually that's one underlying cause branching out, and finding it is exactly where AI already saves time today.

In practice, that's rarely forty separate problems: usually one underlying cause branches through the entire suite. Finding that cause among forty separate error messages is exactly the kind of legwork where AI already concretely saves time today.

What makes the manual process so slow

An engineer manually going through failure analyses opens each failing test report separately, reads the stack trace, looks for a pattern, and tries to reconstruct which change triggered it. With a small suite, that's manageable. With hundreds of tests per run, it becomes a full day's work, and exactly the kind of work where fatigue leads to missed patterns.

In modern CI/CD teams, the bottleneck is often not running tests, but understanding why they fail. AI mainly reduces the time between a red pipeline and an informed decision: fix, rerun, or block.

Instead of forty separate reports, a team gets a summary: "36 of the 40 failures share the same stack trace and point to yesterday's change in the payment module. The remaining 4 appear unrelated." That shift, from forty tasks to five, is where the time saving lies.

What it doesn't replace

Where this pays off the most

• Large regression test suites that run hundreds of tests after every release.
• Teams with limited QA capacity who need to quickly prioritise which failure deserves attention first.
• CI/CD pipelines where fast triage makes the difference between a release going ahead or blocking unnecessarily.
• With recurring flaky tests, AI can recognise patterns: the same timeout, the same endpoint, or the same environmental factor that keeps reappearing.

An example from practice

After a release, 47 tests failed in a client's overnight regression run. Manual triage would normally take half a day. With AI-driven clustering, the team had a summary within minutes.

Instead of 47 separate reports, an engineer got three concrete investigation tasks. Cluster A turned out to be a forgotten database migration in the payment flow; cluster B was temporary instability at the external vendor, not a real regression. Total triage time dropped from half a day to just over an hour.

What we watch for when rolling this out

• Validate the first suggestions manually, even when they look plausible. An AI cluster that happens to be right isn't proof the system understands the cause.
• Keep a log of wrong suggestions. Patterns in what the system clusters incorrectly are themselves valuable information.
• Use it as a starting point for triage, not the endpoint. The final call (fix, rerun, block) stays a human decision.