Predicting churn: A primer for product managers
Collaborating with your data science team
Customer churn is a major challenge for any business that relies on recurring revenue. Losing customers means losing revenue, growth, and reputation. That's why many businesses use machine learning models to predict which customers are likely to churn and take actions to retain them.
Building a churn prediction model is not a simple task. It requires choosing the right features that can capture the behavior and preferences of your customers. Features are the inputs that your model uses to make predictions. The quality and quantity of your features can have a huge impact on the accuracy and interpretability of your model.
How can you collaborate with your data science team to select the best features for your churn prediction model? In this article, I’ll share some tips on how to work with data scientists on feature selection.
Define the problem and the objective
The first step is to define the problem and the objective of your model. What are you trying to predict? What are your business goals and constraints? For example, you may want to predict which customers will churn in the next month, and use that information to design targeted retention campaigns.
Having a clear problem definition and objective will help you and your data science team align on the scope and expectations of the project. It will also help you communicate the value proposition and success criteria of your model to your stakeholders.
Identify potential features
The next step is to identify potential features that are relevant to your problem and objective. You can start by asking data scientists what data they need to perform exploratory analysis and identify potential predictive features. Don't limit them initially. Let them explore different data sources and variables that may be relevant to your problem and objective. For example, they may look at customer demographics, transaction history, product usage, feedback, support tickets, etc.
You can also use your own domain knowledge and research to find out what features are commonly used or recommended for churn prediction. For example, some of the common features for SaaS churn prediction are:
Customer tenure: How long has the customer been using your service?
Recency: When was the last time the customer interacted with your service?
Frequency: How often does the customer interact with your service?
Monetary value: How much revenue does the customer generate for your business?
Contract type: What kind of subscription plan does the customer have?
Service quality: How satisfied is the customer with your service performance and support?
These are just some examples of possible features. You may have other features that are specific to your business domain or use case.
Review potential features
Once you have a list of potential features, you need to review them together with your data science team. As a product manager, you can provide context on which ones make business sense to focus on based on your knowledge of the product and customers. For example, you may know that some features are more important for certain customer segments or markets than others.
The data science team needs to evaluate the quality and availability of each feature. How reliable and complete is the data source for each feature? How easy or difficult is it to collect and preprocess the data?
Quantify predictive power
It is from this point that your role as a product manager in this process is done and the data science team can take over driving the car.
After reviewing potential features, the team needs to quantify their predictive power for churn. This means measuring how much each feature contributes to explaining or predicting customer churn. Data scientists can use various statistical or machine-learning techniques to do this. For example, they can use:
Correlation analysis: This technique measures how linearly related two variables are. You can use this technique to check how each feature correlates with customer churn or with other features.
Information gain: This technique measures how much information a feature provides about customer churn. You can use this technique to check how much each feature reduces the uncertainty or entropy of customer churn.
ANOVA: This technique compares the means of two or more groups of customers based on a feature. You can use this technique to check how different values of a feature affect customer churn.
Feature importance: This technique ranks features based on their contribution to a machine learning model's performance. You can use this technique to check how much each feature improves the accuracy or precision of your model.
Quantifying predictive power will help you identify which features have the strongest signal for churn prediction. It will also help you avoid using features that have no or negative impact on your model's performance.
Remove low-power features
Based on the results of quantifying predictive power, you can discuss removing features that have low or negative predictive power for churn. This will help you simplify your model and reduce the risk of overfitting or multicollinearity. Overfitting means that your model fits too well to the training data but fails to perform well on new or unseen data. Multicollinearity means that some features are highly correlated with each other and may cause instability or redundancy in your model.
However, before removing any feature, you need to consider possible interactions or non-linear relationships between features. Sometimes, combinations of weaker features may have stronger predictive power than individual features. For example, you may find that customers who have certain patterns of behavior are more likely to churn.
Balance accuracy vs. interpretability
After removing low-power features, you need to balance accuracy vs. interpretability of your model. Accuracy means how well your model predicts customer churn correctly. Interpretability means how well you can understand and explain how your model makes predictions.
Generally, there is a trade-off between accuracy and interpretability. Models with more features may have higher accuracy but lower interpretability. Models with fewer features may have lower accuracy but higher interpretability.
You need to consider your business goals and constraints when balancing accuracy vs. interpretability.
Confirm the final feature set
The final step is to confirm the feature set that you and your data science team have selected for your model. Make sure that the selected features are reliable, available, scalable, and maintainable. For example, you may want to avoid using features that depend on external data sources that may change or become unavailable in the future.
TL;DR
Feature selection is a crucial step in building a machine-learning model that can predict customer churn. It involves choosing the most relevant and informative features that can help improve the accuracy and interpretability of your model.
To collaborate with data scientists on feature selection, you need to follow these steps:
Define the problem and the objective of your model.
Identify potential features that are relevant to your problem and objective.
Review potential features together with your data science team.
Quantify the predictive power of each individual feature for churn.
Remove low-power features that have no or negative impact on your model’s performance.
Balance accuracy vs. interpretability of your model.
Transform features if needed to enhance the quality and diversity of your data.
Confirm the final feature set that aligns with your business needs and expectations.
The key is to leverage the data scientists’ technical expertise while providing the business perspective and guardrails as the product manager. Collaborate iteratively, not just in a single step.