Detecting and classifying Android malware families remains challenging due to high feature dimensionality, class imbalance, and the high cost of expert-labeled data.
Semi-supervised learning (SSL) offers a way to leverage unlabeled samples, but prior works rarely test whether SSL benefits generalize across classifier types or report statistical significance.
We present a systematic evaluation of pseudo-labeling across six classifiers
(LightGBM, XGBoost, Random Forest, Logistic Regression, MLP, and SVM) on the CICMalDroid 2020 dataset, using five-fold stratified cross-validation and paired t-tests across five labeled ratios (1-20%).
We find that SSL benefit is strongly classifier-dependent:
- SVM shows the largest significant gain (+4.4% accuracy at 5% labels, p = 0.0028),
- LightGBM improves modestly (+0.8 to +1.3% at 2-5% labels),
- while Random Forest is significantly harmed at low label ratios (-3.1% at 1% labels).
Per-class analysis reveals SSL disproportionately benefits the hardest-to-classify families, with Adware F1 improving by +13.8 percentage points versus only +0.8 for the already well-classified Benign class.
We further show that approximately 800 labeled samples (10% of the dataset) yield near-optimal performance across all classifiers.
These findings offer practical guidance on when and with which classifier pseudo-labeling is worthwhile for Android malware classification.