Deep Learning–Based 3D Segmentation and Spatial Mapping for Tropical Heritage Decay Diagnosis: A Framework from Semarang Sites

Abstract

Diagnosing chromatic decay in tropical heritage buildings poses persistent challenges due to rapid surface transformations and limitations in conventional documentation methods. This study introduces a hybrid diagnostic framework integrating unsupervised hierarchical clustering and supervised Random Forest classification to detect and segment chromatic deterioration using photogrammetric RGB point clouds from four heritage sites in Semarang, Indonesia. The unsupervised method operates on HSV-transformed point clouds, while the supervised model is trained on annotated UV maps. Both approaches were benchmarked against expert-verified ground-truth data. Results showed that the unsupervised model achieved an average precision above 85% and an F1-score exceeding 0.83, with strong adaptability to lighting variability and surface heterogeneity. In contrast, the supervised method provided finer class separation but required intensive manual annotation. The discussion highlights the strengths of unsupervised clustering in scalability and automation, and the precision advantages of supervised segmentation in controlled settings. By addressing spectral ambiguity and reducing annotation dependence, the framework supports semi-automated decay mapping suitable for large-scale heritage inventories. This research contributes a transferable and efficient solution for digital heritage diagnostics, especially in humid, data-scarce environments, and offers actionable insights for preventive conservation in climate-sensitive tropical regions.