A Comparative Evaluation of State-of-the-Art Deep Face Recognition Models on the Indonesian Muslim Student Face Dataset

Abstract

Face recognition systems have achieved remarkable accuracy on standard Western-centric benchmarks, yet their performance on demographically diverse populations remains underexplored. This study presents a comprehensive evaluation of six state-of-the-art deep face recognition models on the Indonesian Muslim Student Face Dataset (IMSFD), which features 26,760 face images of 68 identities captured under varying conditions including the presence of hijab and other head coverings. We evaluate FaceNet (VGGFace2 and CASIA-WebFace variants), AdaFace (ResNet-50 and ResNet-100), MagFace (ResNet-100), and ElasticFace-Arc (ResNet-100) under closed-set identification and face verification protocols. Our evaluation encompasses cumulative match characteristic curves up to rank-20, receiver operating characteristic and detection error trade-off analysis, equal error rate, decidability index ($d'$), true accept rate at multiple false accept rate operating points, per-subset and per-identity analysis, inference speed benchmarking, and computational cost profiling including parameter counts, floating-point operations, and GPU memory consumption. Statistical significance is established via bootstrap 95\% confidence intervals and pairwise McNemar's tests. Results show that AdaFace-R50 achieves the highest rank-1 identification rate of 91.70\% (95\% CI: [90.98\%, 92.38\%]), followed by AdaFace-R100 at 91.32\%, while all pairwise model differences are statistically significant ($p < 0.05$). Notably, substantial performance variation across dataset subsets reveals that recognition difficulty is highly dependent on capture conditions, with subset A2 achieving near-perfect accuracy ($\geq$99.40\%) and subset B1 posing the greatest challenge ($\leq$86.05\%). These findings provide empirical evidence for the need to evaluate face recognition systems on diverse, non-Western datasets and highlight the relative strengths of modern margin-based loss functions for challenging demographic groups.