Understanding Survivorship Bias: Why We Must Be Wary Despite Learning from Successful People

It's natural to want to learn from people that seem successful. And it makes sense because some of the most famous success stories are about people who overcame incredible odds and achieved their dreams. But there is one thing wrong with this assumption: survivorship bias. Survivorship bias causes us to see incomplete pictures of reality by only looking at those that have succeeded, not those that couldn't make it - or worse yet, didn't even try. So while we might feel like we're learning something valuable by studying successful people, in reality, we're just seeing an illusion created by a statistical quirk called survivorship bias.

We often hear that successful people have succeeded because they have done something right. They're strong, wealthy, and accomplished, so we assume they know what they're doing. However, this isn't always the case. In a phenomenon called survivorship bias, we only see those who had success in a given area. We don't see the unsuccessful ones- those who tried and failed to make it big or at all - which can lead us to believe that success is easy if you do A, B & C... when really it could be as simple as luck or happenstance. It's important to keep an eye out for survivorship bias even though it seems like common sense not to take advice from someone with no experience!

It's natural to want to learn from the successful, but this can backfire if we don't account for 'survivorship bias.'

In simple terms, this happens when we only look at the'survivors' — those who outperformed the rest, whether they were people, machines, or companies — and draw conclusions based on their characteristics, rather than looking at the entire dataset, which includes those who had similar characteristics but did not perform as well.

The most well-known example of survivorship bias comes from World War II. At the time, the US military commissioned mathematician Abraham Wald to research the best methods for preventing planes from being shot down. Armour might assist, but it couldn't protect the entire plane, and it would be too heavy to fly well. Initially, their strategy was to inspect planes that had returned from combat, determine where they had been struck the hardest — the wings, around the tail gunner, and down the centre of the body – and then reinforce those places.

But Wald realized they'd succumbed to survivorship bias since their study was missing an important piece of the puzzle: the planes that were hit but never returned. As a result, the military planned to armour the planes in precisely the incorrect places. They were looking at bullet holes that indicated where a plane could be damaged and still fly — exactly the spots that didn't need to be reinforced.

Abraham Wald was a Jewish Hungarian mathematician who developed the discipline of statistical sequential analysis and made contributions to decision theory, geometry, and econometrics. His famous statistical work on how to reduce bomber aircraft damage was written during World War II and accounted for the survivorship bias in his calculations.

What is Survivorship Bias?

Survivorship bias is a logical fallacy that occurs when the underlying data set of a statistical study does not represent a random sample of all possible observations. This means that any conclusions drawn from this type of data will be inaccurate, and may lead to false results.

The reason for survivorship bias in statistics is due to the fact that some individuals or groups are more likely to remain in the population than others. For example, if you were studying how many people get cancer, it would make sense to include those who have been diagnosed with cancer as well as those who haven't been diagnosed at all because they might still develop cancer later on. However, if you were studying how many people live past 100 years old, then you should only include those people who live past 100 years old because people who die at younger ages would not be included.

The survivorship bias can also affect the way that certain statistical tests are performed, but it is usually most prevalent in medical studies which involve diseases or treatments where some patients have died as a result of their condition. For example, if you were creating a clinical trial for cancer treatment, there might be certain criteria in order to determine who is allowed to join the study. For example, they may only include people who are able to afford the treatments or those that do not have any other illnesses besides their type of cancer.

The survivorship bias can also put into question studies about things like airplane crashes, where the events that occurred in the past are analysed and studied to prevent future incidents. However, because some of these occurrences might be related to something like pilot error or a mechanical malfunction which cannot occur again (such as an event that was caused by weather conditions), it may cause survivorship bias when trying to determine if certain safety protocols should be used to prevent future incidents.

When it comes to investments, survivorship bias can affect things like mutual funds where the underlying portfolio may not be diversified enough or does not represent a truly random sample of all possible scenarios (which can result in ignoring certain markets). Survivorship has also caused issues in studies involving specific types of portfolios that are only designed for long-term investments, such as buy-and-hold strategies or low turnover funds.

How does Survivorship Bias work?

When society focuses on successful people, a more subtle form of survivorship bias emerges. Our attention is frequently drawn to persons who succeed "against the odds" or who "take great risks."

For example, a number of today's billionaires, such as Bill Gates and Mark Zuckerberg, have attained success despite never attending or completing university, a fact that has gotten a lot of press. In 2011, Silicon Valley entrepreneur Peter Theil started an ongoing program that awards $100,000 to young entrepreneurs who want to drop out of school (the article featured a cartoon with the lines "College is for suckers").

There are many other examples of survivorship bias in everyday life. There are plenty of other successful companies out there that were just as big at one point but no longer exist today because their founders got bored or ran into trouble. This is called 'the Winner's Curse.' The lesson here is clear: if you want to be a success, don't get complacent - take risks! This principle applies to all areas of life, not just business.

Another example - if you're feeling lonely it may be tempting to focus on your friends who seem happy with their lives. The intense focus on the good that happens around the lives of friends you may envy may cause you to ignore the friends who are experiencing rough patches in their lives. But, for the survivorship-biased narrative in your brain to succeed, it’s important for your focus to just remain on only the tangible good that happens in your friends’ lives. It only takes a moment to break the simulation that survivorship bias can carefully construct - the lives of those around us aren’t as black and white as the supposed narratives in our heads attempt to convince us so. The main message of this? Don't let survivorship bias affect your judgement - make sure that the people in your life who might be having problems know they're loved and supported too!

It's easy to see why stories like this appeal to people. Hearing about successful people who beat the odds gives me hope: if they can earn wealth without going to university, I can, too. However, this is yet another instance of survivorship bias. A quite different picture emerges when you consider all those who do not attend college, rather than simply the successful instances. Graduates had an employment rate of 88 percent in 2018, while non-graduates had a rate of 72 percent. A graduate's median yearly pay was £34,000 ($43,000), whereas a non-graduate was £24,000 ($30,000). Although attending university is not required to be wealthy, looking at the stats rather than the survivors reveals a more complete picture.

In fact, if you closely study the logic of those terms, the danger of basing your perspective of the world on individuals who have 'fought the odds' or succeeded in taking 'huge risks' becomes evident. Such individuals must be unrepresentative of the larger picture, and hence emulating them should be avoided. After all, if everyone was succeeding by taking a big risk, the stakes couldn't have been that high, and the odds couldn't have been that frightening.

We're tempted to believe that success is based on specific traits that can be replicated. It may be worth keeping in mind that success for others is sometimes a matter of luck.

Why should we be wary of Survivorship Bias?

It is vital to be aware of survivorship bias and understand how it might affect your judgment and decision-making in order to ensure that one is exercising critical thinking and making the best decisions possible for oneself (Trying to make decisions your future self will be proud of? Check out this article on The Backwards Law). Survivorship bias can affect you in a variety of ways, so being aware of it might help you make better product decisions, team decisions, or scientific results. Although it is natural for humans to develop biases, taking the time to remove them is vital to guarantee that we make the best decision possible.

Survivorship bias (AKA the "Selection Effect") is the inclination to think that survivors are better or more likely to be right than people who didn't survive. Survivorship bias can affect many areas of life; including education, where students may presume that dropouts are less successful than graduates; finance, where investors may think stocks which have gone up in price over time will continue to go up; and health care, where patients may not want to try treatments with low survival rates because they assume they won't work.

The problem with this assumption is that it doesn't account for what causes people or companies or products to fail--or succeed.

Popular examples of survivorship bias

“We should install armour everywhere except the engine and cockpit because planes returning from battle have bullet holes everywhere except the engine and cockpit.

While this case has no clear economic relevance, we'll start here because it's widely regarded as the origin of the concept of survivor bias.

Allied troops intended to add defensive armour to their warplanes during WWII. They couldn't put armour on the entire plane due to a lack of resources. As a result, experts had to determine which locations were the most vulnerable to assault and would benefit the most from extra security.

They initially observed damage on planes which had been shot down but made it back home to decide where the armour should go. The fact that these planes had no bullet wounds in the engine or cockpit led to the obvious conclusion that they should equip the planes everywhere except the cockpit and powerplant.

Fortunately, mathematician Abraham Wald pointed out the fault in their strategy - they were only looking at the planes that had arrived safely at their destination. Because the gunshot holes were not deadly to the planes, they were not taken down. Instead, he suggested that the military add armour to the regions where the surviving planes had no bullet holes.

Wald coined the term "survivorship bias" after studying the planes that had crashed, saving numerous lives in the process.

“I'll be a millionaire like Steve Jobs, Bill Gates, and Mark Zuckerberg, who all dropped out of college.”

A fast Google search for "successful founders who dropped out of college" will provide some of the world's most famous names. Jobs, Gates, and Zuckerberg are all instances of successful entrepreneurs who had a vision, took a risk, and miraculously succeeded.

However, by linking their achievement solely to hard work, we overlook one key fact: for every successful college dropout, there are hundreds, if not thousands, who were not so fortunate.

We place the founders on pedestals because they worked hard, but there were also many coincidences that helped them succeed. According to studies, the vast majority of America's most successful businesses — 94 percent to be exact — attended college.

The assumption that a college diploma isn't required for success is a prime example of survivorship bias. While it may not be appropriate for everyone, it is critical to consider all available information contextually before making a decision.

"I'll understand how to be successful if I read the biographies of the world's most successful entrepreneurs."

"Successful People's Morning Routines," "The One Thing Jeff Bezos Says Made Him Successful," "The Six Characteristics All Billionaires Have in Common." How many of these articles did you read? I consider myself guilty of having opened these and numerous others.

We enjoy the idea of being able to replicate our heroes' achievements by studying about them. The issue is that these pieces, and even deep-dive bios, don't provide all of the information we need to replicate their accomplishment. We ignore variables that are not evident to the ordinary reader and make decisions based on insufficient information.

While it's difficult to discover books about the hundreds of people who undoubtedly attempted and failed to build Amazon before Bezos, it's vital to remember that striving to replicate the success of the success won't always provide identical results. When hard work isn't enough, circumstances take control.

"I'll be successful if I model my company after Warby Parker."

"The Netflix of tomorrow." "The Uber of [insert industry name here]." You've probably heard these terms in regard to hot new startups. Only two-thirds of all firms survive two years, half of all businesses survive five years, and one-third of all enterprises survive ten years, according to the US Small Business Administration.

The odds are stacked against you, and just because you model your firm after a successful one doesn't mean you'll automatically assimilate the same level of success.

This type of survivorship bias drives many entrepreneurs to try to fit their company into a model that isn't appropriate for their current market, audience, or stage of growth.

When launching a new business, take inspiration from the organizations you respect, but evaluate the industry to see how you can improve your concept.

"Because my product is better than theirs, I'll succeed."

Did you know that TiVo is still in operation? This digital video recorder was one of the first on the market and is still considered one of the best today. It was so popular at its peak that it became a verb. When we know they've been using DVR for at least five years, most of our parents are still telling us they TiVo'd "The Amazing Race."

When a better product can't beat the convenience, brand loyalty, or market flood, TiVo is a fantastic example. Don't believe that just because you have a higher-quality product or service, your customer will choose you automatically (case in point, when Sketchers edged out Adidas in market share a mere a couple of years ago).

To avoid the survivorship bias, you must consider other variables that are not immediately obvious.

How to avoid being biased by survivorship bias

People who don't know about survivorship bias can easily fall into the trap of believing one thing and doing another. There are several ways you can avoid this, such as:

Putting yourself in the position of someone who has experienced something you haven't, like a survivor of sexual abuse.
Not basing your decisions too heavily on what other people are doing or saying; remember that they could be biased as well (and use them for information, not decision making).

Anyone can fall into this trap, even those who are aware of survivorship bias. You can avoid it by:

Remembering that your experiences aren't the only ones out there and you don't know everything! Someone else's experience is just as important as yours, so take what they say with a grain of salt.
If someone claims to have an "answer" to a problem, assume they're biased and look for more information about the topic.
Not basing your decisions too heavily on what other people are doing or saying; remember that they could be biased as well (and use them for information, not decision making).

Think about what you don't see

When we think about survivorship bias, we usually think of Abraham Wald, a statistician who studied World War II planes. His Columbia University research group was tasked with figuring out how to better safeguard planes from destruction. The initial strategy was to examine the planes as they returned, determine where they were struck the most, and then reinforce that region.

However, Wald concluded there was a crucial piece of evidence missing: planes that were hit but never returned. Planes that crashed and did not survive had a lot more information to share about the regions that needed to be reinforced the most. Wald's strategy demonstrates how to overcome survivorship bias. Don't focus solely on what you can see. Consider anything that attempted to follow the same route but failed. Try to figure out their narrative - you may learn more from their failures more so than the success of others.

It's tough to consider survivorship bias when faced with examples of success. It's not natural to pause, ponder, and consider what the base rate of success is and whether you're dealing with an outlier or the expected outcome. You have a blind spot if you don't know the true odds, especially if you don't know if what you're looking at is an example of survivorship bias.

Next time Steve Jobs tells you his story, immortalised in articles and YouTube clips, think about all the people who came before who tried and failed. But don’t let these moments of reflection limit you. Understanding survivorship bias isn't an excuse for not acting, but it is a necessary tool for cutting through the noise and comprehending the world. If you're going to do something, be sure you're prepared.

Survivorship bias is the idea that we only see those who have succeeded in a given field or endeavour, not all of them. This can lead us to believe that success is easy if you do A, B & C... when really it could be as simple as luck or happenstance. It's important to keep an eye out for survivorship bias even though it seems like common sense not to take advice from someone with no experience!

Conclusion

Survivorship bias is a logical fallacy that we all need to be aware of. It’s especially important to be mindful when making decisions based on sample pieces of data and stories sold to us through media, as these narratives told to us may not be an accurate representation of reality. By being conscious of survivorship bias and its effects, we can make more informed choices and avoid costly mistakes. In this post, we outlined some tips for avoiding survivorship bias in our own lives. Did these tips help you? Let us know in the comments! And if you want to learn more about how to identify and mitigate other common fallacies, check out some of our other articles on authority bias, confirmation bias and memory bias.