Optimizing Breast Cancer Classification: SVM and Random Forest with Hybrid Hyperparameter Tuning and Feature Selection

Abstract

Breast cancer remains one of the leading causes of cancer-related mortality among women worldwide, underscoring the urgent need for early, accurate, and reliable diagnostic support systems. This study proposes an optimized breast cancer classification framework using Support Vector Machine (SVM) and Random Forest (RF) models enhanced through hybrid hyperparameter tuning and feature selection. The Breast Cancer Wisconsin (Diagnostic) dataset, comprising 569 samples with 30 numerical features derived from Fine Needle Aspirate (FNA) examinations, was utilized in this research. Feature selection was conducted using Random Forest feature importance to identify the most relevant diagnostic attributes and reduce dimensionality. Hybrid hyperparameter tuning was implemented using GridSearchCV combined with 5-fold cross-validation to obtain optimal model configurations. Model performance was evaluated using accuracy, malignant-class recall, confusion matrix analysis, and Receiver Operating Characteristic–Area Under the Curve (ROC–AUC). Experimental results show that the optimized SVM model achieved significant improvements in accuracy, recall, and ROC–AUC compared to baseline models, indicating enhanced sensitivity and discrimination capability, while the Random Forest model maintained stable performance with marginal gains after optimization. These findings highlight the critical importance of systematic optimization strategies in improving diagnostic safety and reducing false negatives, thereby contributing to the development of more reliable and clinically applicable machine learning-based medical decision support systems.