Multi-FOM: A Unified Multimodal Deep Learning Framework for Intelligent Pre-Harvest Papaya Classification and Crop Valorisation

Abstract

Intelligent pre-harvest crop assessment has become an essential component of precision agriculture, where accurate crop quality evaluation enables informed decision-making and sustainable crop management. However, existing multimodal frameworks often exhibit limited data diversity and insufficient cross-modal semantic interaction, reducing the reliability of intelligent crop valorisation. Address these limitations, this paper proposes a Multimodal Focal-Mamba Optimized Valorisation Model (Multi-FOM), a unified deep learning framework integrating papaya image data and IoT sensor data for intelligent pre-harvest crop analysis. Initially, the Artificial Diffusion-based Image Augmentation Module (ADM) generates diverse and semantically consistent papaya image samples to improve dataset variability and model generalization. Subsequently, the Unified Multimodal Crop Harmonization Module (UMCH) pre-processes and harmonizes heterogeneous multimodal inputs to generate consistent crop representations. The Focal-Mamba ExcelFormer Tabular Representation Network (FMETRNet) employs a dual-stream architecture to extract complementary visual and IoT sensor features. These representations are integrated through the Perceiver-Q Cross-Modal Fusion Network (PQCF-Net), which performs latent cross-modal alignment and query-guided semantic refinement to produce unified crop features. The fused representation is further processed by the Deep Crop Intelligence Learning Module (DCIL) to capture crop health, fruit quality, maturity, environmental responses, and nutrient information. Finally, the Alpha-SiLU Activated Blue-Eared Hedgehog Optimized Lite Transfer Learning Model (αS-BHOT) performs adaptive feature optimization and intelligent crop classification to generate comprehensive crop valorisation outcomes. Experimental results demonstrate 99.67% classification accuracy, precision of 99.53%, recall of 99.61%, 99.72% F1-score, and a minimum MSE of 0.0123, confirming Multi FOM as an effective decision-support framework for next-generation precision agriculture.