Telemarketing is one of the convenient and effective methods of selling products and services to the customer. If not targeted to the right customer, these telemarketing calls might be perceived as irritating which might instead decrease the company’s value. Using a Portuguese bank telemarketing dataset, we implemented various machine learning algorithms to predict the right customer. We utilized an over-sampling method called SMOTE to mitigate the class imbalance problem. The Light Gradient Boosting Machine (LGBM) model on the plain dataset scored highest AUC of 0.80 then the over-sampled dataset, implicating that SMOTE might not add any benefits to complex ensemble tree methods. For model explainability, we implemented the global as well as local explainer to streamline the decision-making process.

The dataset is open-source and can be downloaded from the link. The original paper that open-sourced data can be found at:

Moro, S., Cortez, P., & Rita, P. (2014). A data-driven approach to predict the success of bank telemarketing. Decision Support Systems, 62, 22-31.

The dataset is imbalced consistingof 4,639 'yes' and 36,549 'no'.

The methodology section is sub-divided into following section:

1. Pre-Processing

   • Removed the varibale like 'pdays', 'default' due to the lack of variability.
   • Removed column 'default' as suggested by author.
   • Encoded the nominal and ordinal value to one-hot encoding and ordinal encoding respectively.
   • The conitnuous data were standardized using Z-score.

2. Train-Test Split & Cross Validation

   • 90% Train, 10% Test
   • Train data is further divided into 10 Fold Cross-Validation.
   • Conducted statrified split to preserve the same ratio of target class.

3. SMOTE

   • SMOTE is utilized to tackle class imbalance problem.
   • Generates synthetic data to balance the class on training set.

4. Models Used

   • Random Forest (RF), Logistic Regression (LR), Extra Tree Classifier (ET), Xtream Gradient Boosting Machines (XGBM), Light Gradient Boosting Machines (LGBM), CatBoost (CB).

5. Evaluation Metrics

   • Precision (PR), Recall (RC), ROC-AUC (Area Under Curve), F1-ratio, Accuracy (ACC), Matthews Correalation Coefficient (MCC)
   • AUC score will be used to select the best model.

• The experiment are divided into two parts. One with the plain data (no SMOTE) and other with SMOTED data.

Note: * denotes that the value is highest among two different experimentation across various evaluation metrics.

• The LGBM Model without SMOTING performed best among others.
• This implicates that for complex ensemble models SMOTE might not add any benefits (Elor & Avrbucg-Elor, 2022).

• In general, the threshold is set to be 0.5, and so was ours, which means that the predicted probability of an instance that is greater than 0.5 is flagged as positive and less than 0.5 as negative.
• We ran several experiments on the LGBM model to find the optimal cut-off value.

• From figure, we found that 0.19 are our optimal model.
• We trained new model with that threshold and evaluated on the test data.

• The Precison and Recall has incresed along with F1 and MCC.

• Feature Importance
• Local Interpretable Model-Agnostic Explanations (LIME)

• Deployed the app that gives marketers ability to input customer's data in order to get the probability of either buying or rejecting the sales call.
• The app also shows the features, with the help of LIME, that effects the consumers decision.

• The deployed app can be accessed via link

Even in the era of social media marketing, telemarketing still stands out with its own advantages. Our results show that the plain model outperformed the SMOTED model. The LGBM model achieved an AUC score of 0.8055 on the test set. Our comparison validates previous findings that SMOTE might not be valuable when used in more complex ensemble tree algorithms (Elor and Averbuch-Elor 2022). However, it might add some value to a simple tree or linear model. Explainability remains at the center of any predictive modeling when comes to business decision-making. We implemented a feature importance plot for global explainability. However, the feature importance plot doesn’t give us the ability to further investigate a specific customer. Therefore, LIME is implemented to explain every single observation. To our knowledge, there hasn’t been any research done in a telemarketing domain where LIME was implemented to explore each instance. A robust predictive model with interpretability enables identifying the right customer and avoiding annoying calls to uninterested customers. The interpretable methods also help to understand the consumers’ intention so that the business personal can make data-driven unbiased decisions. Such insight will help company to attract the right customer eventually contributing to the company’s growth.

Plese refer to the requirements.txt file.

Elor, Y., & Averbuch-Elor, H. (2022). To SMOTE, or not to SMOTE? Arxiv.Org. http://arxiv.org/abs/2201.08528