OPTIMIZING XGBOOST FOR CUSTOMER CHURNPREDICTION IN E-COMMERCE USINGGRIDSEARCHCV AND STRATIFIEDCROSS VALIDATION
DOI:
https://doi.org/10.47701/m609h512Keywords:
Customer Churn Prediction, E-Commerce, GridSearchCV, Hyperparameter Optimization, Stratified Cross Validation, XGBoostAbstract
Customer churn prediction is important for e-commerce
companies because identifying customers who are likely to
leave can help companies develop more targeted retention
strategies. This study compares baseline and optimized
XGBoost models for customer churn prediction using
GridSearchCV and Stratified Five-Fold Cross-Validation.
The experiment used an e-commerce churn dataset
containing 5,630 customer records, 18 predictor variables,
and one binary target variable. Data preprocessing
included median imputation, Min-Max scaling for
numerical features, Ordinal Encoding for categorical
features, and removal of the CustomerID identifier. The data
were divided into training and testing sets using an 80:20
stratified split. For model optimization, preprocessing,
SMOTE, and XGBoost were integrated into a pipeline, with
SMOTE applied only to the training portion of each cross-
validation fold. GridSearchCV evaluated 216
hyperparameter combinations, with the best configuration
achieving a mean cross-validation F1-Score of 0.8747 ±
0.0155. On the independent test set, the baseline model
achieved 99.11% Accuracy, 97.37% Precision, 97.37%
Recall, 97.37% F1-Score, and 99.92% ROC-AUC, while the
optimized model achieved 98.85% Accuracy, 97.84%
Precision, 95.26% Recall, 96.53% F1-Score, and 99.74%
ROC-AUC. The optimized model slightly improved
Precision but did not outperform the baseline on the other
evaluation metrics. These findings show that
hyperparameter optimization does not necessarily improve
overall test-set performance and should therefore be
assessed using an independent test set.
References
Andespa, R., Sadik, K., Suhaeni, C., & Soleh, A. M. (2025). Evaluating Random Forest
and Xgboost for Bank Customer Churn Prediction on Imbalanced Data Using Smote
and Smote-Enn. Media Statistika, 18(1), 25–36.
https://doi.org/10.14710/medstat.18.1.25-36
Azhar, M., Saikhu, A., & Buliali, J. L. (2025). Optimization of Predictive Ensemble Model
for Customer Churn Through the Integration of SMOTE-ENN and GridSearchCV.
2025 International Conference on Data Science and Its Applications (ICoDSA),
1216–1220. https://doi.org/10.1109/ICoDSA67155.2025.11157631
Brito, J. B. G., Bucco, G. B., Heldt, R., Becker, J. L., Silveira, C. S., Luce, F. B., &
Anzanello, M. J. (2024). A framework to improve churn prediction performance in
retail banking. Financial Innovation, 10(1). https://doi.org/10.1186/s40854-023-
00558-3
Chen, N., & Hu, M. (2023). Frontiers in Service Science: Data-Driven Revenue
Management: The Interplay of Data, Model, and Decisions. Service Science, 15(2),
79–91. https://doi.org/10.1287/serv.2023.0322
Chen, T., & Guestrin, C. (2016). XGBoost: A Scalable Tree Boosting System. Proceedings
of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and
Data Mining, 785–794. https://doi.org/10.1145/2939672.2939785
Fazli, S., & Mohammadi, M. J. (2026). Leveraging gradient boosting machine learning
models to predict customer churn in digital health platforms. Discover Health
Systems, 5(1). https://doi.org/10.1007/s44250-026-00387-y
Gala, D. M., Pawar, B., Band, G., Dua, P., Mohanty, B. R., & Patil, B. V. (2024). Enhancing
Predictive Accuracy for Customer Churn in Digital Banking: A Multi-Model
Analysis. 2024 3rd Edition of IEEE Delhi Section Flagship Conference (DELCON),
1–5. https://doi.org/10.1109/DELCON64804.2024.10866346
Gunasekara, N., Pfahringer, B., Gomes, H., & Bifet, A. (2024). Gradient boosted trees for
evolving data streams. Machine Learning, 113(5), 3325–3352.
https://doi.org/10.1007/s10994-024-06517-y
Hakyemez, T. C., & Adar, O. (2024). Testing the Efficacy of Hyperparameter Optimization
Algorithms in Short-Term Load Forecasting. http://arxiv.org/abs/2410.15047
Imani, M., Joudaki, M., Beikmohammadi, A., & Arabnia, H. R. (2025). Customer Churn
Prediction: A Systematic Review of Recent Advances, Trends, and Challenges in
Machine Learning and Deep Learning. Machine Learning and Knowledge Extraction,
7(3), 1–38. https://doi.org/10.3390/make7030105
Liang, Z. (2023). Predict Customer Churn based on Machine Learning Algorithms.
Highlights in Business, Economics and Management, 10, 270–275.
https://doi.org/10.54097/hbem.v10i.8051
Maan, J., & Maan, H. (2023). Customer Churn Prediction Model using Explainable
Machine Learning. 11(1), 33–38. http://arxiv.org/abs/2303.00960
Manzoor, A., Atif Qureshi, M., Kidney, E., & Longo, L. (2024). A Review on Machine
Learning Methods for Customer Churn Prediction and Recommendations for
Business Practitioners. IEEE Access, 12, 70434–70463.https://doi.org/10.1109/ACCESS.2024.3402092
Mouli, K. C., Raghavendran, C. V., Bharadwaj, V. Y., Vybhavi, G. Y., Sravani, C.,
Vafaeva, K. M., Deorari, R., & Hussein, L. (2024). An analysis on classification
models for customer churn prediction. Cogent Engineering, 11(1).
https://doi.org/10.1080/23311916.2024.2378877
Ouf, S., Mahmoud, K. T., & Abdel-Fattah, M. A. (2024). A proposed hybrid framework to
improve the accuracy of customer churn prediction in telecom industry. Journal of
Big Data, 11(1). https://doi.org/10.1186/s40537-024-00922-9
Peng, K., Peng, Y., & Li, W. (2023). Research on customer churn prediction and model
interpretability analysis. PLOS ONE, 18(12), e0289724.
https://doi.org/10.1371/journal.pone.0289724
Rahman, M. M., & Khan, M. K. (2023). Mechanisms By Which Ai-Enabled Crm Systems
Influence Customer Retention and Overall Business Performance: a Systematic
Literature Review of Empirical Findings. International Journal of Business and
Economics Insights, 03(01), 31–67. https://doi.org/10.63125/qqe2bm11
