Comparison of Prediction Models for Heart Disease Data: Logistic Regression, Random Forest and Extreme Gradient Boosting

Abstract

This paper aims to use predictive models such as the Logistic Regression (LR), Random Forest (RF) and Extreme Gradient Boosting (XGBoost) to predict the onset of cardiovascular disease based on relevant risk factors and make a comparison of the performance of these three models. First, this paper examined the statistical significance of the initial variables and the linear correlations between individual variables. To prepare the data for modeling, this paper first normalized all numerical and binary variables to ensure they were on a comparable scale. Dimensionality was then reduced using Principal Component Analysis (PCA), with the goal of simplifying the feature space while retaining meaningful variance. For the Random Forest (RF) model, parameter tuning prioritized minimizing RMSE, though the selected configuration also led to strong classification accuracy. XGBoost was optimized through a stepwise grid search combined with stratified k-fold validation, helping to maintain consistent performance across data splits. When comparing Logistic Regression, RF, and XGBoost, the latter consistently achieved the best classification results-likely due to its ability to capture complex and non-linear feature relationships. While these early findings are encouraging, especially for XGBoost, further validation on broader and more diverse datasets will be important before drawing firm conclusions about its clinical utility.