Part 4 of 9 in How to Think Better: Harnessing the Power of Counterintuition← Part 3Part 5 →
This chapter explains why statistical models and diverse crowds can outperform individual experts in forecasting. It helps readers decide when to trust models, crowds, experts, or AI, especially in finance, medicine, cricket, and investing.
- Individual experts are losing ground to models and diverse crowds in many forecasting tasks.
- Experts remain especially valuable for causal understanding, complex diagnosis, patient communication, and adapting to new rules.
- Aggregated forecasts and models often outperform charismatic individuals in uncertain prediction.
- Tracking expert records can reveal when confidence reflects strong evidence or mostly storytelling.
- AI can provide an additional crowdsourced voice by comparing models, prompts, studies, and historical data.
Mauboussin describes the “expert squeeze.” In many domains, the value of individual experts is declining because statistical models and the wisdom of diverse crowds now outperform them. The expert’s edge remains in areas that require deep causal understanding or rapid adaptation to new rules, but it is shrinking in pure forecasting.
India has a strong tradition of guru culture—in finance, medicine, cricket, and spirituality. This makes the expert squeeze both more relevant and harder to accept. We are culturally inclined to look for a wise individual rather than a well-calibrated process.
Indian Examples
Stock market tipsters on television and Telegram groups regularly underperform simple index funds over multi-year periods, yet they retain large followings. The narrative is compelling; the long-term track record is not. Cricket experts confidently predict series outcomes; the historical record shows that simple models based on recent form, conditions, and team strength are often as good or better.
In medicine, individual doctors still add enormous value in complex diagnosis and patient communication. But for many routine risk predictions (for example, the probability of complications after a standard procedure or the likelihood of success with a particular fertility treatment given age and diagnosis), well-calibrated models now perform at least as well as average specialists. The best doctors use these models rather than ignore them.
In early-stage investing, a diverse group of angels or a simple scoring model based on team quality, market size, and traction often matches or exceeds the performance of any single “star” investor over a large sample of deals.
How to Use the Expert Squeeze
- Ask whether the domain is primarily one of prediction under uncertainty or one of causal intervention. Models and crowds do better in the former; skilled experts still dominate the latter.
- When seeking forecasts, prefer aggregated or model-based estimates over single charismatic experts.
- Use experts for framing the problem and identifying the right variables, then let data and models do the prediction.
- Track the track records of experts you follow. Most people do not. Doing so is humbling and useful.
Using AI as a Crowdsourced Expert
Modern AI systems can be viewed as a form of compressed collective intelligence. They have been trained on vast amounts of text, including research papers, historical outcomes, and expert writing. They are not infallible, but they are a useful additional voice.
- For forecasting questions, ask multiple AI models or the same model with different prompts and look for consensus. This is a cheap form of the wisdom of crowds.
- “What do the best-available studies and historical data say about the success rate of Indian angel investments in deep-tech startups that raise their first institutional round within 18 months of founding? Provide ranges, not point estimates. Note the quality of the evidence.”
- Use AI to identify when an expert’s track record is strong versus when it is mostly storytelling. Ask it to summarize published track records where available.
Frequently asked questions
When do models and crowds outperform experts?
Models and crowds often do better in prediction under uncertainty, including stock forecasts, cricket outcomes, routine medical risks, and early-stage investing.
When are individual experts still more useful than models?
Experts remain valuable for causal intervention, deep causal understanding, rapid adaptation to new rules, complex diagnosis, and patient communication.
How should I use experts and models together?
Use experts to frame the problem and identify important variables, then use data and models for prediction. Prefer aggregated estimates over one charismatic expert.
How can AI help evaluate expert forecasts?
Ask multiple AI models, or use different prompts, and look for consensus. AI can also summarize published track records and identify whether an expert's record is strong or mostly storytelling.