The Planning Fallacy
Eliezer Yudkowsky joins us from Overcoming Bias, an econblog devoted to human rationality and the cognitive psychology of mistakes. And if you think that’s interesting, you should see his day job.
The Sydney Opera House is among the most epic construction overruns of all time. Initially slated to be completed in 1963 for $7 million, it was finally completed in 1973 for $102 million. But don’t look down on them: Experiment shows that, deep in our hearts, we all have an inner home improvement contractor.
One study (I’m not going to include the bibliography in this post; but you can find it all here) asked students when they thought they’d complete their academic projects. Specifically, the students were asked for times by which they thought it was 50%, 75%, and 99% probable they would’ve finished already.
Care to guess how many students finished on or before their estimated 50%, 75%, and 99% certain delivery dates? If so, better guess now, because I’m about to tell you:
- 13% of the students finished their project by the time they had assigned a 50% probability;
- 19% finished by the time assigned a 75% probability;
- and only 45% (less than half!) finished by the time of their 99% probability.
This phenomenon is more generally known as the “planning fallacy”. Roughly, the planning fallacy is that people think they can plan.
A clue to the underlying problem was uncovered by researchers who found that asking subjects for their predictions based on realistic “best guess” scenarios, versus asking subjects for their hoped-for “best case” scenarios, produced indistinguishable results.
See, when you ask someone for a “realistic” scenario, they envision every event happening the way they think is normal – which usually means, just like I planned it. No unexpected delays, no unforeseen catastrophes – what people envision by default is the best-case scenario.
Reality, it turns out, usually delivers results somewhat worse than the “worst case”. When you ask people to envision everything that can possibly go wrong, their schedule gets a lot closer to reality – but still not close enough.
Unlike most cognitive biases, the planning fallacy has a simple remedy – though I’ll warn you, you’re not going to like it.
The same researchers asked another group of students to describe highly specific plans for their Christmas shopping – plans that described where, when, and how. Another group was simply asked when they expected to finish their Christmas shopping. The first group, with the detailed plans, expected to finish shopping more than a week before Christmas – the second group expected to finish an average of 4 days before Christmas – and in reality, both groups finished an average of 3 days before Christmas. That’s right: detailed planning made the students more optimistic.
Why? Another study, done in Japan, helps to illuminate the answer: A group of Japanese students expected to finish their essays, on average, 10 days before the deadline. They actually finished 1 day before deadline. Asked when they’d finished previous essays, they said: “One day before deadline.”
You see, you do have a reliable source of information about how well you’ll do. It’s how well you did last time. But the more details you visualize, the more chance you have to be optimistic – to visualize everything going exactly as planned – instead of remembering how long it took last time, when things didn’t go as planned.
A similar finding is that experienced outsiders, who don’t know all the details and all the special reasons why this project is bound to do unusually well, but who do have a lot of experience on projects in that area, tend to be a lot less optimistic and a lot more accurate than the actual planners and implementers.
The “inside view” is when you generate your predictions and time estimates by thinking about all the unique details of how it’s going to go this time – planning where, when, and how.
The “outside view” is when you deliberately avoid thinking about the special features of this project – deliberately avoid fine-tuning your estimate – and remember how long it took you to finish broadly similar projects in the past.
The inside view has its uses. There’s a certain amount to be said for, like, actually planning things ‘n stuff. But figuring out how long your project will really take, is not one of those uses. For schedule estimates, the outside view beats the inside view, hands down, every time.
So there’s a reliable way to fix the planning fallacy, if you’ve got the strength to stomach it. Just ask how long it took you the last few times, without considering any of the special reasons this project will be different. Better yet, ask an experienced outsider how long broadly similar projects have taken (and be sure not to tell them the details).
You’ll get back an answer that sounds hideously long, and clearly reflects no understanding of all sorts of important particulars. This answer is true. Deal with it.
This has been Eliezer Yudkowsky, Research Fellow of the Singularity Institute for Artificial Intelligence. Some say he may be the mad scientist who destroys the world, others say he may be the mad scientist who saves it, a few say he’s just plain mad – but not many people say he’s boring. He’s still blogging at Overcoming Bias; check it out!
