Base-Rate Neglect: Ignoring the Odds That Matter

Key Takeaways

• Base rate neglect means underweighting the general frequency of an outcome and overweighting case-specific details.
• Kahneman and Tversky showed it with the lawyers-and-engineers study, where people ignored the pool composition.
• Bayes theorem says you must combine prior odds with new evidence, but people often drop the prior.
• In investing it fuels chasing rare winners and trusting forecasts that the historical record contradicts.

What It Is

A base rate is the underlying frequency of something in the relevant population. Base rate neglect, also called the base rate fallacy, is the tendency to ignore that frequency and judge a case mostly by its specific, individuating details.

The classic demonstration comes from Daniel Kahneman and Amos Tversky. Participants read personality sketches drawn from a pool that was, say, 70 percent engineers and 30 percent lawyers, then guessed each person's profession. People leaned almost entirely on whether the sketch sounded like an engineer or a lawyer, and largely ignored the pool composition. A description that sounded balanced should have defaulted toward the 70 percent group, but it rarely did.

The Intuition

Specific detail is concrete and emotionally vivid. A base rate is an abstract number that feels distant from the case at hand. When the two conflict, the vivid detail wins. This is driven by the representativeness heuristic: you judge how likely something is by how well it matches a stereotype, instead of how common it actually is.

In markets, the individuating detail is the exciting story about one company, fund, or trade. The base rate is the boring statistic about how often companies like this succeed, how often active funds beat their index, or how often a chart pattern plays out. The story is loud. The base rate is quiet. Investors who skip the base rate are flying without the most reliable instrument available.

How It Works

The correct procedure is Bayesian: start with the prior probability, the base rate, then update it with new evidence in proportion to how diagnostic that evidence is. People tend to anchor on the new evidence and forget the prior entirely.

Note one important nuance from later research, including work by Gerd Gigerenzer. When the same problem is framed in natural frequencies, such as 30 out of every 100, people use base rates far better than when it is framed in percentages or probabilities. The bias is real but partly a function of how the numbers are presented. The practical takeaway is to translate any decision into plain counts, then ask what the base rate says before the story gets a vote.

Worked Example

A test screens stocks for fraud. It is 90 percent accurate: it flags 90 percent of true frauds and clears 90 percent of clean firms. A stock you hold gets flagged. How worried should you be?

Most people guess around 90 percent likely fraud. They neglect the base rate. Suppose only 1 in 100 firms is actually fraudulent. Out of 1,000 firms, 10 are frauds and 990 are clean. The test flags 9 of the 10 frauds, and it falsely flags 10 percent of the 990 clean firms, which is 99 firms. So a flagged stock comes from a pool of 9 plus 99, or 108 flagged firms, of which only 9 are real frauds.

That means a flagged stock has roughly a 9 in 108 chance of being fraud, about 8 percent, not 90 percent. The low base rate of fraud overwhelms the test's accuracy. An investor who dumps the stock in panic at the flag has neglected the base rate and likely overreacted to a mostly false alarm.

Common Mistakes

1. Judging a stock only by its story. A great narrative says nothing about how often similar companies succeed. Always ask for the base rate of that category.

2. Believing forecasts over track records. A confident, detailed prediction can feel more reliable than the historical hit rate of such predictions. The record is the base rate.

3. Overreacting to a single signal. As the fraud test shows, an alarming flag against a rare event is usually a false positive. Combine the signal with the prior odds.

4. Chasing rare winners. The few stocks that returned 100 times are vivid, but the base rate of any random stock doing so is tiny. Sizing bets as if lightning is common is costly.

5. Using percentages instead of counts. Framing decisions in raw frequencies, like 9 out of 108, makes base rates harder to ignore than abstract percentages.

Frequently Asked Questions

What is base rate neglect in simple terms? Base rate neglect is ignoring how common something is and judging it instead by specific details. You focus on the vivid story in front of you and forget the underlying odds.

How does base rate neglect affect investment decisions? It leads investors to chase exciting stories and overreact to single signals while ignoring how often such outcomes really happen. As the fraud-test example shows, neglecting a low base rate can turn a mostly false alarm into a panicked sale.

What is a real-world example of base rate neglect? Treating a 90 percent accurate fraud flag as a 90 percent chance of fraud, when the rarity of fraud means a flagged stock is actually fraudulent only about 8 percent of the time.

How can investors avoid base rate neglect? Translate decisions into plain counts, then ask what the historical frequency says before weighing the specific story. Combine prior odds with new evidence rather than dropping the prior.

How is base rate neglect different from the conjunction fallacy? Base rate neglect is ignoring the underlying frequency of an outcome. The conjunction fallacy is overrating combined events because they fit a stereotype. Both arise from the representativeness heuristic.

Sources

1. The Decision Lab. "Base Rate Fallacy." https://thedecisionlab.com/biases/base-rate-fallacy
2. Gigerenzer, G. (1991). "How to Make Cognitive Illusions Disappear." https://sites.stat.columbia.edu/gelman/communication/Gigerenzer1991.pdf
3. Farnam Street. "Bias from Insensitivity to Base Rates." https://fs.blog/mental-model-bias-from-insensitivity-to-base-rates/
4. Springer, Review of Philosophy and Psychology. "On the Reality of the Base-Rate Fallacy." https://link.springer.com/article/10.1007/s13164-023-00712-x

Disclaimer

This article is educational content only and is not financial advice. Nothing here is a recommendation to buy, sell, or hold any security. Consult a licensed advisor before making investment decisions.