5 Common Cognitive Biases and How to Avoid Them
With examples from sports and product management
Cognitive biases are systematic errors in the way that we understand and use information, and they can affect our decision-making and problem-solving abilities as product managers. In my experience, I’ve found five cognitive biases that product managers routinely face. But there are MANY more. Click the below image for a deep dive.
Confirmation bias
Confirmation bias is a tendency that can affect how people see and understand new information. It can make people only pay attention to things that support their existing beliefs or ideas, and ignore things that don't. This can impact how people make decisions, form opinions, and do research. For example, you could be loyal to Apple as a customer and believe that the Macbook M1 Pro can outperform any PC laptop. So you research and find data that only confirms your beliefs while disregarding data that goes against your beliefs.
While researching this bias, I found a great piece where Chris Trapasso, an NFL draft analyst at CBS Sports, had Justin Jefferson ranked as the 64th overall prospect in 2020. He concluded that Jefferson’s success in college was primarily due to the fact that he played slot receiver. In other words, he confirmed his conclusion that Jefferson’s college production may not translate to production in the NFL because of the position Jefferson played. Jefferson's rookie season with the Minnesota Vikings was outstanding (and every subsequent season so far at the time of this writing) and it proved that he could excel outside of the slot position. Further analysis of Jefferson’s LSU career would have uncovered that his role as a slot receiver was related to LSU’s team needs and not necessarily to Jefferson’s skills and abilities.
How about an example from the PM world? Let’s say you are a product manager at an early-stage startup. After launch, you receive lots of positive feedback from early adopters and some negative feedback about the product. You focus on the positive feedback and decide that you are on the path to product-market fit and that the negative feedback just isn’t worth prioritizing right now. Yet it turns out that as more users adopted the product, later adopters had the same negative feedback as some of the early adopters, and now with more frequency. Because the positive feedback confirmed your existing belief that your product was on the right trajectory, you now have a retention problem and are not so sure you are on the path to product-market fit.
To avoid confirmation bias, product managers should:
Consider all data and feedback available
Test ideas and assumptions with experiments and A/B tests
Seek feedback from people who may disagree with you or have different experiences
Survivorship Bias
Survivorship bias is when you only look at the winners and forget about the losers. It makes you think that some things are more important or easier than they really are because you only see the successes and not the failures.
For example, imagine you want to start a startup, and you read stories of people who got rich by starting their own companies. You may think that starting a startup is easy and that all you need is a good idea, some hard work, and a little bit of luck. But this is classic survivorship bias because you are only seeing the success stories and not the many other people who failed or lost money. In reality, starting a startup is much harder and riskier than it looks.
Here is another example from sports. If you saw this chart, you may conclude that quarterbacks tend to 2x their productivity after the age of 40. But this chart is missing important context. There is only one quarterback who “survived” to play at the age of 43 and 44 and in those two seasons he threw for 4,633, and 5,316 yards, respectively. Ahem, Tom Brady. (I highly recommend reading the full post from Caleb Smith here. )
What does this look like for a PM?
Imagine you are a B2C e-commerce product manager conducting an analysis of your best-selling products and attempting to determine the success factors behind them. You focus on the products that have the highest sales numbers and customer ratings and try to identify common characteristics among them. However, you ignore the products that have experienced a decline in sales or have been discontinued.
By focusing exclusively on the products that have survived and disregarding those that have failed, your analysis is vulnerable to survivorship bias. You may miss valuable insights and factors that caused some products to fail, which could help prevent your failure in the future for your successful products. You can avoid survivorship bias by gathering feedback from dissatisfied or churned customers and understanding why they stopped using the product.
Optimism Bias
Founders and product manager generally approach their work with positivity and optimism, believing that if they put in the work they will add value for their customers and company. If there is any chance for a startup or new product to succeed, at a minimum founders and product managers alike have to believe that they will succeed. But there is a danger here to being too optimistic and falling prey to optimism bias.
Optimism bias is defined as the difference between a person’s expectation and the outcome that follows.
In her book The Optimism Bias: A Tour of the Irrationally Positive Brain, cognitive neuroscientist Tali Sharot explains that we experience optimism bias more when we think the events are under our direct control and influence. In other words, we believe we have the unique skills and ability to change the outcome.
We may think to ourselves: “I will not fail, because I have the best product; I will not run out of money, because I have a solid plan for growth. There is an illusion of invulnerability which can lead to disappointment.
One way to check optimism bias is by conducting a premortem.
A premortem is a technique to overcome optimism bias by imagining that a project has already failed and identifying the possible reasons for the failure. This technique helps to anticipate potential problems, avoid blind spots, and plan for contingencies.
Availability Bias
Availability bias is a cognitive error that occurs when we rely on information that is more easily recalled or accessible, rather than on more accurate or representative information. This can lead us to overestimate the importance or frequency of certain events, problems, or user needs, based on our personal experience or exposure.
We see this type of bias in sports fans every day. For example, your favorite team may have won a match in a convincing fashion. The manager of the club has used a formation in the match not used since the beginning of the season. You, as a fan, attribute the success to the change in tactics, forgetting that the team had lost previously using the same formation.
In product management, we are always looking for the latest data to help us make decisions. The problem that can occur here is relying too much on the latest information and discarding previous data. For example, you may have plenty of customer feedback that your product’s reporting tools need to be more robust. But now, generative AI and ChatGPT are the talk of tech. You may decide that you need to prioritize a chatbot in your product out of FOMO (fear of missing out). This is an example of availability bias affecting decision-making.
As mentioned before, to combat availability bias, product managers need to be balanced in their approach to the data available and validate hypotheses through assumption testing and experimentation.
Anchoring bias
Anchoring bias is a tendency for people to rely heavily on the first piece of information they receive when making decisions. This initial information becomes a reference point, or "anchor," for their decision-making process. People then make adjustments to their initial estimation based on this anchor, but these adjustments tend to be insufficient and biased toward the initial estimation. This means that if the initial information was wrong or biased, it can lead to inaccurate decisions.
Let’s look at this hypothetical. Your CEO gathers the product team in a meeting and announces a big brand new project that is going to be a game-changer for the company (also optimism bias). This project will open up a new business line and has the potential to triple revenue. The CEO has asked you to lead the project. As you conduct your research, you start to find evidence that this project may have difficulty finding product-market fit. But your opinion is anchored to your CEO’s initial estimation of success for the project and so you downplay the data. This is how anchoring bias can lead to poor decisions.
Avoiding cognitive biases and errors
To avoid these cognitive biases and errors here are some tips to keep in mind:
Consider all data and feedback available
Test ideas and assumptions with experiments, A/B tests, confidence intervals, and regression analysis
Evaluate your reasoning and logic
Seek feedback from people who may disagree with you or have different experiences
Seek feedback from dissatisfied or churned customers
Conduct a premortem to anticipate potential problems, avoid blind spots, and plan for contingencies
Great post!