In the late 1970s, Daniel Kahneman was part of a team developing a new curriculum for Israeli high schools. They had been working for a year when Kahneman asked the group to estimate how long the project would take to complete.
The estimates ranged from 18 months to 2.5 years. Then he asked Seymour Fox, the most experienced member of the team, a different question: of the teams he knew who had undertaken similar curriculum projects, what fraction had actually completed them? And for those that did, how long did it take?
Fox’s answer was stark. He couldn’t think of a single similar project that had been completed in under seven years. Many were never finished at all.
The curriculum took eight years. The team had estimated, at most, two and a half.
This is the planning fallacy in one story. The bias is not subtle or marginal. It is large, systematic, and resistant to the kind of partial awareness that most people apply to it.
The 1979 Paper and What It Actually Said
Kahneman and Tversky introduced the planning fallacy in a 1979 paper that described a cluster of related cognitive tendencies: people focus on the specific features of the task at hand, construct an optimistic scenario for its completion, and largely ignore the base rate — the historical distribution of outcomes for similar tasks.
This “inside view” versus “outside view” distinction became central to understanding why the bias is so persistent. The inside view asks: what do I know about this specific project, and what’s the most likely way it will unfold? The outside view asks: what happened to other projects of this type?
The inside view produces vivid, detailed forecasts that feel accurate. The outside view produces statistical distributions that feel impersonal and difficult to apply. When the two conflict, people almost always favor the inside view — even when they know the outside view is more accurate.
This is not irrationality in a simple sense. The inside view feels right because it uses all the specific information you have. The problem is that specific information is seductive and distributional information is dull. You can imagine your project; you can’t imagine the distribution of projects like yours.
Kahneman elaborated on this framework in a 1994 paper with Dan Lovallo, published in Psychological Review, which introduced the concept of “reference class forecasting” as the prescriptive correction. To estimate accurately, you need to anchor on the outside view — the distribution of outcomes for the reference class of similar projects — and then adjust from there based on specific features that genuinely distinguish your project.
Buehler, Griffin, and the Replication Evidence
Roger Buehler and Dale Griffin at the University of Waterloo ran a series of studies in the 1990s that replicated and extended the original findings with more granular experimental designs.
Their 1994 paper in the Journal of Personality and Social Psychology studied students estimating how long academic projects would take. The students consistently underestimated — not once, not in unusual cases, but reliably across tasks. More tellingly, when asked to generate a “realistic” scenario for how the project might go — including possible obstacles and setbacks — they still underestimated.
The critical finding from Buehler and Griffin’s work: thinking about past experiences with similar tasks produced only marginal improvement in forecast accuracy. When students were explicitly prompted to recall how long similar projects had taken, their forecasts moved slightly toward accuracy but remained substantially optimistic. The historical information was available but didn’t override the vivid imagined scenario.
A follow-up study by Buehler, Griffin, and Ross looked at whether people learned from their own prediction errors. The finding was discouraging: people generally attributed their past overruns to exceptional, unrepeatable circumstances. “That project ran late because of the technical complications.” “This one will be different.” Each project’s overrun was treated as a special case rather than as base-rate evidence.
This “unique circumstances” attribution is a key mechanism keeping the planning fallacy alive even in experienced planners. You don’t update your base-rate estimate because you interpret each overrun as an anomaly, not as a signal about your estimation tendencies.
Flyvbjerg: The Megaproject Evidence
Bent Flyvbjerg at Oxford’s Saïd Business School has studied the planning fallacy at the largest possible scale: infrastructure megaprojects.
His dataset, accumulated over decades and described in papers and his 2023 book How Big Things Get Done (with Dan Gardner), covers thousands of large infrastructure projects globally — dams, bridges, tunnels, railway lines, power plants, the Olympics. The findings are striking.
Across his dataset, the average cost overrun for large infrastructure projects is around 45%. Schedule overruns are similarly prevalent. And these overruns happen at roughly the same rate regardless of the era, the country, or the level of project management sophistication applied.
Flyvbjerg argues this isn’t primarily explained by bad luck or technical complexity — it’s explained by systematic optimism bias in the initial planning phase, sometimes combined with strategic misrepresentation (deliberately understating costs to get projects approved). The inside view dominates the planning process even for teams with decades of project experience.
His prescription is reference class forecasting as a formal method: before finalizing a project estimate, identify the reference class of similar projects, calculate the distribution of actual outcomes in that reference class, anchor your estimate to the distribution, and adjust only for features of your specific project that are genuinely and demonstrably different from the reference class.
This sounds straightforward but is organizationally and psychologically difficult. It requires finding an appropriate reference class (harder than it sounds), resisting the impulse to treat your project as exceptional (very hard for project proponents), and accepting a forecast that may be unwelcome (hardest of all in organizations where pessimistic forecasts face political resistance).
Flyvbjerg has worked with government bodies in the UK, Denmark, and elsewhere to integrate reference class forecasting into infrastructure project appraisal. The results in contexts where it’s been properly implemented show meaningfully better forecast accuracy — not perfect, but substantially better than unanchored inside-view estimates.
Software Engineering: A Domain That Has Measured the Problem Intensively
Software estimation research provides some of the most granular data on the planning fallacy in professional knowledge work, partly because the industry has been obsessed with the problem for decades.
Capers Jones, who studied software projects extensively, found that software projects overrun their original estimates by an average of 70–80% in time. Barry Boehm’s work on software cost models in the 1980s and 1990s documented the “cone of uncertainty” — the idea that estimates early in a project carry massive error ranges that only narrow as work progresses and unknowns are resolved.
The Standish Group’s CHAOS Report, published annually since 1994, has tracked software project outcomes consistently. The percentage of projects delivering on time and on budget varies by how you count it and how the methodology has evolved, but the central finding has been consistent: late delivery and overrun are the norm, not the exception, across the industry.
Research on agile and iterative methods suggests that breaking projects into smaller increments improves local estimation accuracy — sprint-level estimates are more accurate than release-level estimates, which are more accurate than project-level estimates. This is consistent with the inside-view mechanism: smaller, more concrete tasks are easier to visualize completely, reducing the amount of unknown work that gets excluded from the imagined scenario.
But even sprint-level estimates in agile contexts tend to overrun. The planning fallacy operates at every level of granularity.
What Actually Works: The Evidence on Correction
If awareness doesn’t reliably fix the planning fallacy, what does?
Reference class forecasting. The empirical evidence is strongest here. Studies of its application in infrastructure project appraisal (particularly Flyvbjerg’s work) and in experimental settings (Buehler, Griffin, and colleagues) show that anchoring estimates to historical base rates produces meaningfully better accuracy than inside-view estimation alone.
The challenge is that reference class forecasting requires data — a record of how similar tasks or projects have actually gone. For individual knowledge workers, this data doesn’t exist until you build it through systematic planned vs actual tracking. Creating your own reference class database is the personal version of what Flyvbjerg recommends at the project level.
Pre-mortem analysis. Gary Klein’s pre-mortem technique — imagining that the project has failed and working backward to identify why — has some empirical support for improving estimate accuracy by surfacing risks that the inside view excludes. It’s not a substitute for reference class data, but it partially corrects the “best-case scenario” problem.
Explicit buffer rules. The 50% rule — estimating tasks at 1.5x your naive estimate — is a rough but defensible heuristic for applying a blanket correction. It’s less accurate than task-specific reference class data but more accurate than unanchored inside-view estimates. For situations where you lack historical data, it’s a reasonable starting point.
Decomposition. Breaking tasks into smaller sub-tasks and estimating each separately tends to produce more accurate aggregate estimates than estimating the task as a whole. This is partially because decomposition forces you to think through the full scope rather than stopping at a vague gestalt impression. It doesn’t eliminate the planning fallacy, but it reduces it.
The Honest Limits of the Research
A few caveats worth naming.
Most planning fallacy research uses student samples doing academic tasks. Generalization to professional knowledge workers doing complex, interdependent project work is reasonable but not fully established.
Some researchers have questioned whether the planning fallacy is truly distinct from general optimism bias or whether it’s better understood as a rational response to incentive structures (projects get funded if estimates are optimistic, so optimistic estimates are strategically rational even if they’re inaccurate). Flyvbjerg’s concept of “strategic misrepresentation” addresses this — the bias may be partly motivational, not purely cognitive.
The replication landscape for behavioral science research generally is complex. The core phenomenon — systematic underestimation of task and project duration — is one of the most robustly replicated findings in the field. Specific mechanistic claims (exactly why it happens, exactly how large the effect is in a given context) are somewhat more contested.
What’s not in serious dispute: people consistently underestimate how long things will take, the bias persists across experience and expertise, awareness alone doesn’t reliably fix it, and the most effective correction involves anchoring to historical data rather than relying on imagined scenarios.
Applying This to Your Planning Practice
The research points toward a specific practice: track what you plan, track what actually happens, and build a personal database of reference-class estimates over time.
This is what the Reality Check Loop (see The Complete Guide to Planned vs Actual Time Analysis) is designed to support. The Capture phase builds your reference class data. The Compare phase surfaces the variance. The Calibrate phase applies the reference class to update your planning defaults.
You are implementing, at the individual level, exactly what Flyvbjerg recommends at the project level: replace inside-view estimation with outside-view reference class anchoring, adjusted for the specific features of the task at hand.
The planning fallacy is real, large, and persistent. But it’s not immutable. The research that established its existence also identified the tools for correcting it. Those tools are now accessible to individual knowledge workers with a consistent tracking practice and an AI assistant to help with the analysis.
Related: The Complete Guide to Planned vs Actual Time Analysis — Why Planned vs Actual Analysis Fails
Suggested tags: planning fallacy, Kahneman, reference class forecasting, Flyvbjerg, time estimation science
Frequently Asked Questions
-
Who coined the term 'planning fallacy'?
Daniel Kahneman and Amos Tversky first used the term 'planning fallacy' in a 1979 paper, and Kahneman elaborated on the concept in a 1994 paper with Dan Lovallo. The concept was popularized for general audiences in Kahneman's 2011 book, *Thinking, Fast and Slow*. The term describes the tendency to make overly optimistic predictions about time, costs, and risks while neglecting the base rate of similar past tasks.
-
Is the planning fallacy the same as optimism bias?
They overlap but aren't identical. Optimism bias is a broader cognitive tendency to overestimate positive outcomes across domains. The planning fallacy is a specific manifestation of optimism bias in the context of time and cost estimation for tasks and projects. The planning fallacy specifically involves neglecting historical base rates — you've done similar things before and they took longer, but you disregard that history when estimating the current task. Not all optimism bias involves this reference class neglect.
-
Does knowing about the planning fallacy help you avoid it?
Somewhat, but less than most people expect. Awareness alone — knowing that you tend to underestimate — produces modest improvement at best, and the improvement is often short-lived. The more effective correction is structural: using reference class forecasting (anchoring estimates to historical data on similar tasks), building in explicit buffers based on category-level variance data, and tracking planned versus actual time to calibrate future estimates. Knowing about the bias is insufficient without a system that uses that knowledge.