An incident review has one job: to leave the organisation knowing something it did not know before. Almost everything that goes wrong with reviews comes from an unstated second job, which is to establish who was responsible.
Hindsight makes the path obvious, and it was not
Afterwards, the sequence looks like a straight line. The warning sign is visible in the graph, the risky change is obviously risky, and the decision that led to the outage looks careless.
At the time, that graph was one of forty, the warning was indistinguishable from a dozen similar fluctuations that meant nothing, and the change looked like the eleven others that week that were fine. Nothing marked the signal that mattered until it mattered.
This distortion is well described outside software, in accident investigation and in the study of expert decision-making, and it is strong enough that knowing about it does not remove it. A review that does not actively correct for it will reliably conclude that somebody should have seen what nobody could have seen.
Counterfactual language teaches nothing
"Should have checked", "could have rolled back", "failed to notice" all describe a world that did not happen. They feel like analysis and they contain no information about the world that did.
The productive replacement is to ask what actually guided the action. What was on the screen. What the tooling made easy and what it made hard. What the normal practice was. Whether anyone had seen this failure mode before. Those answers describe the system that produced the decision, which is the only thing that can be changed.
There is rarely a root cause
The singular is the problem. Substantial failures in systems of any complexity require several conditions to hold at once: a latent defect, a configuration that made it reachable, a check that did not cover it, a piece of documentation that was out of date, and somebody under time pressure.
Remove any one of them and there is no incident. Calling one of them the root cause is a decision about where to stop looking, and the investigation usually stops at the last human involved, because a person is a satisfying place for a chain to end.
Naming contributing conditions in the plural produces more useful work, because each one is separately addressable and several will be cheaper to fix than the one that got the title.
Blameless is a method, not a kindness
The case for it is instrumental. The information needed to understand an incident is held mostly by the people closest to it, and people do not volunteer information that will be used against them. A review that punishes produces reviews that are careful, incomplete and useless, starting with the next one.
Aviation and medicine reached this conclusion the expensive way and built reporting systems around it. The finding is not that people never behave badly. It is that a system which cannot hear about its own failures cannot correct them, and hearing requires safety.
What blameless does not mean
It does not mean nothing is ever anyone's responsibility. Safety practice distinguishes honest error, which is what most incidents involve; behaviour that knowingly accepts an unjustified risk, which calls for a conversation about why that felt normal; and reckless disregard, which is a management matter and is not absolved by process.
Stating the distinction openly matters, because teams that hear "blameless" without it either disbelieve it or conclude that nothing counts, and both undermine the method.
Separate the timeline from the analysis
Establish what happened, in order, with times, before anyone discusses why. Mixing them means the narrative gets shaped by the emerging explanation, and details that do not fit the story quietly fail to make it into the document.
Run it within days, while memory is intact. Have it facilitated by somebody who is not in the reporting line of the people involved, which is a small structural change that does more for candour than any statement of values.
Fewer actions, actually done
The standard output is a list of a dozen items, of which two are completed. That pattern is so consistent that a review producing twelve items has mostly produced a document.
Three, with named owners and dates, tracked in the same place as other work, beats twelve. And one specific item is worth privileging: reviews are the largest source of new alert rules, and the article on deleting them describes what happens when that source runs unchecked. Adding a rule at a review should carry the same question as any other alert, which is whether it would be worth an interruption.
What we cannot verify
The safety-science findings referenced here come from other industries, and their transfer to software operations is an argument rather than a demonstration. The completion rate for action items is a widely shared practitioner observation, not a measured statistic, and we present it as such. Claims that particular incident management products improve learning come from their vendors.
The short version
- Nothing marked the signal that mattered until it mattered.
- Counterfactual language describes a world that did not happen and teaches nothing.
- Ask what guided the action: the screen, the tooling, the normal practice.
- Naming one root cause is a decision about where to stop looking.
- Blameless is instrumental: a system that cannot hear about failures cannot correct them.
- Three action items with owners beat twelve, and new alert rules need the same test as any other.