Blameless postmortems: template, example, and tracking action items to done Intermediate

This page answers: how to write a postmortem that assigns no blame yet keeps accountability, and how to make sure action items actually get done.

When to write one

A postmortem is a written record of an incident: its impact, the actions taken to mitigate it, the causes, and what must be done so it does not recur. The SRE book lists common triggers:

• Users see downtime or degradation beyond a threshold.
• Data loss of any kind.
• On-call had to intervene (rollback, rerouting traffic…).
• Resolution time exceeded a threshold.
• A monitoring failure (which usually means the incident was discovered manually).

What blameless means

Per the SRE book, a blameless postmortem focuses on identifying contributing causes without indicting any individual or team for bad behavior. The reason: “you can’t ‘fix’ people, but you can fix systems and processes”.

Blameless is	Blameless is not
Asking “what system let this error reach production?”	Pretending nobody did anything
Accurately describing who did what and when, based on the information they had at the time	Removing names from the timeline to be “polite”
The people involved remain responsible for telling the story and fixing it (accountability)	Nobody has to do the action items
Accepting that human error is normal and designing guardrails	Ignoring deliberate violations (that is a management matter, not a postmortem matter)

Language shapes culture. The SRE Workbook gives a bad example: calling an engineer’s actions “careless ignorance”. A rewrite should describe conditions, not character:

Bad:   "Binh deployed without testing and caused the incident."
Good:  "Release v2.31 passed CI and was deployed at 13:58 UTC. The test suite
        had no load test for the export endpoint, so synchronous rendering of
        400 ms/request was not detected before production."

Postmortem template

Copy the template below. All timestamps use UTC so they line up across teams and logs.

# Postmortem: <short incident name> — <YYYY-MM-DD>

Status: Draft | In review | Approved
Author: <single owner>   Reviewers: <...>
Incident doc: <link>   Severity: SEV1 | SEV2 | SEV3

## 1. Summary (3-5 sentences)
What happened, who was affected, how long it lasted, how it was fixed.

## 2. Impact (with numbers)
- Impact window: HH:MM → HH:MM UTC (N minutes)
- Users / requests affected: <absolute number and %>
- SLO: consumed <x>% of the period's error budget
- Data: lost or corrupted yes / no; if yes, how it was reconciled
- Revenue / orders / support tickets (if measurable)

## 3. Timeline (UTC)
| Time | Event | Source |
|---|---|---|
| HH:MM | Triggering change (deploy, config...) | CI / audit log |
| HH:MM | First alert / user report | Alertmanager |
| HH:MM | Incident declared, IC = ... | Incident channel |
| HH:MM | Mitigation: ... | |
| HH:MM | SLI back to normal | Dashboard |

## 4. Root cause and contributing factors
- Trigger: the triggering event.
- Root cause: the pre-existing condition in the system that let the trigger cause an incident.
- Contributing factors: why it was not caught by review / tests / canary / alerts.

## 5. Detection and response
- How was it detected, and after how long? Did the alert reach the right people at the right severity?
- How was it mitigated, and how long from declaration?

## 6. Lessons learned
### What went well
### What went poorly
### Where we got lucky
(conditions that should have made the incident worse but did not happen)

## 7. Action items
| # | Item | Type | Priority | Owner | Due | Ticket |
|---|---|---|---|---|---|---|
| 1 | ... | prevent / detect / mitigate | P0-P2 | @... | YYYY-MM-DD | ... |

## 8. Appendix
Graphs, queries, reports, PR links.

The Where we got lucky section and the action item categories come from the SRE Workbook’s example postmortem. The Workbook also uses investigate and repair categories; the three main categories below are enough for most teams.

Type	Question	Example
prevent	How do we make this error impossible?	Lint rule banning *Sync in HTTP handlers
detect	How do we find out sooner?	Alert when event loop delay max is above 200 ms for 5 minutes
mitigate	Next time it happens, how do we reduce the damage?	Feature flag for the export endpoint; one-command rollback runbook

A filled-in example (hypothetical, illustrative numbers)

A scenario built from Demo 1, not a real incident:

• Trigger: release v2.31 adds an invoice export endpoint that renders HTML synchronously, about 400 ms per request.
• Root cause: a CPU-bound handler runs on the main thread, so every export request blocks every other request in the same process.
• Contributing factors: no load test for the new endpoint; no event loop delay alert; the dashboard only shows average latency.
• Got lucky: the incident happened outside peak hours, and the rollback had no migration, so it was reverted in 2 minutes.

Action items:

1. mitigate, P0: kill switch for the export endpoint.
2. prevent, P1: move rendering to a worker thread / job queue.
3. detect, P1: event loop delay max alert.
4. prevent, P2: add a load test step to the release checklist for endpoints that process large data.

“5 Whys” and its limits

5 Whys asks “why?” repeatedly until it reaches a root cause. It is useful to push past the first layer of symptoms:

Checkout slow → why? Event loop blocked for 400 ms
  → why? The export endpoint renders HTML synchronously
  → why? No convention for running CPU-bound work off the main thread
  → why? ...

The limits, per John Allspaw (“The Infinite Hows”, 2014):

• A single linear chain, while an incident usually has several parallel branches of causes.
• “Why” easily slides into “who”, breaking the blameless spirit.
• A root cause is something investigators choose, not something found ready-made; stopping at the fifth “why” is a convention, not a truth.

Safe usage: run 5 Whys for each contributing factor, ask “how” instead of “why”, and stop when you reach something fixable in the system.

Tracking action items to done

The SRE Workbook quotes Ben Treynor Sloss: to users, a postmortem without follow-up actions is indistinguishable from no postmortem. The Workbook also notes that without a formal tracking process, action items are often forgotten, leading to repeat incidents.

Loading diagram…

1. Each item is a ticket in the real backlog, with a link back to the postmortem.
2. A single owner with a deadline. “The backend team” is not an owner.
3. Specific and verifiable. “Improve monitoring” is wrong; “alert when event loop delay max is above 200 ms for 5 minutes, routed to checkout on-call” is right.
4. Do not rely on “everyone be more careful”. A good prevent item changes the system: lint, tests, guardrails, automation.
5. Review overdue P0/P1 items periodically (for example in a weekly operations meeting).
6. Postmortems must be reviewed. SRE book: an unreviewed postmortem might as well never have existed.

Postmortem review checklist

1. Impact has numbers (duration, requests, % of error budget), the timeline is complete and in UTC.
2. Trigger, root cause and contributing factors are distinguished.
3. No sentences blaming individuals and no emotional language; describe conditions, not character.
4. There is a “Where we got lucky” section.
5. Each contributing factor has at least one action item; prevent, detect and mitigate are all present.
6. Each action item: one owner, a deadline, a ticket, verifiable, not relying on “be more careful”.
7. Action item priority matches the impact.
8. The postmortem is shared widely, and items are tracked on a schedule until closed.

Next: Common mistakes.