How to run a decision retrospective in 30 minutes (and what the data reveals about your judgment)

The retro nobody runs

Engineering teams run sprint retros every two weeks.

PMs review OKRs at quarter-end.

Almost nobody reviews the decisions themselves.

Did you make the right call against the evidence you had at the time?

Six months later, can you tell the difference between a good decision that turned out poorly and a bad decision that got lucky?

Most teams can't.

The 30-minute format

• Pull every decision logged in the last 30 to 90 days that has hit its review date.
• For each, read the original confidence level and the evidence cited.
• Score the outcome: did the underlying assumption hold?
• Tag with one of three labels — Held, Didn't Hold, Right By Accident.
• Write one sentence on what the decision teaches you about your judgement.

The three categories

Held — the assumption was correct, the decision worked, and you can re-derive the same call from current evidence. These are the decisions you should celebrate but not over-learn from.

Didn't Hold — the evidence shifted, the customers churned, the metric definition changed. The decision is now wrong. The interesting question is whether the failure was foreseeable.

Right By Accident — the bet paid off, but the assumption that justified it turned out to be false. You picked the right move for the wrong reason. This is the most dangerous category because it reinforces bad decision-making with positive outcomes.

What the third month reveals

Run the retro three months in a row.

By month three, a pattern emerges.

Most PMs find one of three things.

Either their high-confidence calls fail more often than their low-confidence calls (overconfidence pattern).

Or their 'right by accident' rate is climbing (skill-luck confusion).

Or there's a specific evidence type — usually sales calls — that consistently leads to decisions that don't hold.

The pattern is what changes the next decision.