Billionaire college dropouts: Why smart leaders avoid “survivorship bias”
- Survivorship bias occurs when we ignore the possibility that important data has been left out of the selection process.
- A few famous college dropouts became billionaires — but most are unsuccessful, a fact obscured by survivorship bias.
- To combat survivorship bias, leaders must seek out the “silent evidence.”
During World War II, the U.S. Army Air Forces faced a devastating dilemma. Enemy forces were downing aircraft faster than they could be manufactured and the losses among airmen were staggering. U.S. officers knew they needed to reinforce the aircraft with armor, but adding too much weight would render them less maneuverable and limit effective range. They needed to determine which parts of the aircraft to protect with armor and where they could safely leave it off.
They studied aircraft that returned from battle and found certain parts were statistically more likely to be hit than others. Obviously, the solution was to add armor to these areas. Obvious — and wrong.
Thankfully, the Army Air Forces also enlisted the help of Abraham Wald, a brilliant mathematician who took a teaching position in the States to escape rising anti-Semitism in Vienna. Looking over the data, he concluded the opposite. The parts of the aircraft in need of armor were those that showed the least chance of being hit, namely the engines and cockpit. Why? Because their data only considered the aircraft that had survived the operation and returned. They also needed to consider the ones that hadn’t returned and why that may be.
Wald’s keen observation saved thousands of airmen’s lives during the war. It also offers future leaders an important lesson in “survivorship bias.”
The billionaire college dropout
Survivorship bias is a logical error in which we consider the data at hand but ignore its selection process. As a result, we assume the data set is reflective of the group as a whole, and that assumption may blind us to valuable truths and insights. In the example above, only the returning aircraft were considered and that influenced how the military interpreted the dangers.
It need not be as drastic as wartime survival, either. Another example is the billionaire college dropout. Because dropouts like Bill Gates, Steve Jobs, and Mark Zuckerberg are so famous, their stories can suggest that the path to success is to leave college and incubate your passion project. And for some people, it will be.
However, if you seek out a more comprehensive data set, a different story emerges. Research shows that college dropouts, on average, earn less than college graduates and are 70% more likely to be unemployed. Gates and Zuckerberg aren’t the rule; they are the exceptions that outshine the rule.
Leaders must be especially careful of survivorship bias because it can insidiously influence their thinking and decision-making. This can prompt them to make mistakes and take unnecessary risks due to over-optimism, confusing correlation with causation, and acting on unwarranted assumptions.
For instance, the leader of a young startup may aim to become the Netflix or Uber or Whole Foods of her industry. Assuming that the best lessons come from those successful businesses, she carefully studies their strategies and learns from their example. However, this selection process excluded the many failed startups who tried to become the Netflix or Uber or Whole Foods of their industries. Those lessons are then lost to her — lessons that may have helped her steer through situations those successful businesses never encountered.
Blinded by success
For an example of survivorship bias in the business world, look no further than Jack Welch’s influence on the American leader’s mindset. Welch gained his reputation as the CEO of General Electric (GE) between 1981 and 2001. Under his leadership, GE outperformed the Standard & Poor’s 500 index, and its stock price rose 5,600%, from $2.38 the day before he took office to $135.69 on his last.
This meteoric success led many business leaders to assume that Welch’s choices and strategies could be a blueprint for their own success. In other words, their selection process was to pay attention to Welch’s hits while ignoring his misses. And in hindsight, Welch found his success in spite of some incredibly poor decisions.
Take Welch’s style of performance management. He had managers rank teams along a 20-70-10 split. The top 20% were deemed the most productive and received bonuses. The middle 70% were left alone, and the bottom 10% were laid off. Welch gave his system a wholesome-sounding name: the vitality curve. His employees dubbed it “rank-and-yank.”
Blinded by Welch’s success, CEOs soon made rank-and-yank their gospel. Jeffrey Skilling, once considered the next Welch, imposed rank-and-yank at Enron as part of his “survival-of-the-fittest” philosophy. Adam Neumann adopted it at WeWork. Both crashed and burned. Even adopters that didn’t fail, such as Amazon and Microsoft, have gotten into hot water over its use.
Now, none of this is to say that Enron and WeWork failed because of rank-and-yank. (That’s its own kind of logical fallacy.) But it seems unlikely that a management practice with so many painful flaws would have caught on so widely if the adoption criteria were broader than Welch’s luster alone. Thankfully, as the data have come in, the practice has lost its glow.
The same may be said of Welch’s tenure, too. As David Gelles, author of The Man Who Broke Capitalism, explained to NPR: “Fast forward and look at what happens to [the] pensioners. Look what happens to the men and women holding GE stock as it plummets in the years following [Welch’s] departure. Because when he leaves, all of these flaws are fundamentally exposed and Wall Street starts seeing through the charade.”
The sound of silent evidence
The first step in combating survivorship bias’s influence on your thinking and decision-making is to simply be aware of it. Recognize that your sources of data — whether it’s the news, your team, or even yourself — have implemented some sort of selection process. That process may leave you with an image of reality that isn’t as comprehensive as you think.
After that, follow Wald’s example:
Search for the silent evidence. In The Black Swan, Nassim Nicholas Taleb calls the data left out of the selection process “silent evidence.” Wald recognized that the Army Air Forces’ selection process accidentally left out downed aircraft. Once he noticed that, he sought out the silent evidence — those areas on the planes without bullet holes.
Remember to always interrogate your selection process and question what information it may have excluded. Then seek out that evidence as best you can. In the GE case study, it would have behooved leaders to study the results of rank-and-yank at other organizations and research its effects on people who experienced the process, rather than assuming Welch’s success was the only metric needed.
Make reasonable assumptions. Sometimes, the silent evidence is unobtainable. You may not be able to find it or it may not exist yet. At such a point, you’ll need to make assumptions to avoid “analysis paralysis.” But those assumptions should be reasonable.
In Wald’s case, he couldn’t study the downed aircraft directly. He had to assume that the lightly damaged parts were important. Otherwise, the returning aircraft should have had a statistically more even spread of bullet holes. He was correct.
Similarly, it is reasonable to assume that rank-and-yank will have knock-on effects that damage corporate culture and, in time, the bottom line. It degrades trust and lowers collaboration by creating internecine rivalries. Evaluations can be arbitrary and unfair since the system obscures individual contributions through forced ranking. And it’s ripe for exploitation through favoritism and office politics.
Of course, we can never completely avoid survivorship bias. As with other biases, this one is baked into the human experience. But as a leader, you’ll do more for your organization and better for your people if you can learn to resist its temptations.
.jpg){kind=link}