A Multimodal Deep Learning Framework for Prostate Cancer Risk Prediction Integrating Biparametric MRI, Radiology Reports, and Clinical Biomarkers

Author First name, Last name, Institution

• Walid Mohamed, Assiut University
• Ibrahim Abdelhalim, J.B. Speed School of Engineering
• Tayseer Hassan, Assiut University
• Amr Abdelaziz, Faculty of Computer and Information
• Mohammed Badawy, Mansoura University, Urology and Nephrology Center
• Moumen El-Melegy, J.B. Speed School of Engineering
• Asem Ali, J.B. Speed School of Engineering
• Ali Mahmoud, J.B. Speed School of Engineering
• Fatma Taher, Zayed University
• Mohammed A. Ghazal, Abu Dhabi University
• Sohail Contractor, University of Louisville School of Medicine
• Ayman El-Baz, J.B. Speed School of Engineering

Document Type

Conference Proceeding

Source of Publication

Proceedings International Symposium on Biomedical Imaging

Publication Date

4-8-2026

Abstract

Accurate prostate cancer risk stratification requires integration of heterogeneous clinical data sources that exhibit variable quality and complementary diagnostic information. We propose EFNet-SAF (Expert Fusion Network with Staged Adaptive Fusion), a multimodal framework that integrates Biparametric MRI (DWI, T2-weighted), radiology reports, and clinical biomarkers (e.g.,PSA, Age) through an instance-specific adaptive approach. The architecture employs modality-specific expert networks trained independently with pretrained encoders, followed by adaptive fusion that learns patient-specific modality importance. Hierarchical imaging processing exploits spatial correspondence between DWI and T2 sequences, while clinical biomarkers are incorporated through a learned weighting mechanism. Evaluated on 73 prostate cancer cases using 5 -fold cross-validation, EFNet-SAF achieved an AUROC of 0.945 ± 0.06, significantly outperforming single-modality approaches (imaging: 0.842, text: 0.918, clinical: 0.747, all p<0.05) and standard fusion baselines (early fusion: 0.825, late fusion: 0.861). Ablation studies show that each modality adds useful information, and removing text has the biggest effect on performance (Δ AUROC=0.137). The staged training paradigm and adaptive weighting mechanism facilitate resilient multimodal integration on constrained clinical data while preserving interpretability via acquired instance-level modality significance.

DOI Link

10.1109/isbi61048.2026.11515882

ISBN

[9798331577636]

ISSN

1945-7928

Publisher

IEEE

Volume

2026-April

Disciplines

Computer Sciences | Medicine and Health Sciences

Keywords

clinical decision support, medical imaging, Multimodal fusion, prostate cancer, radiology reports

Scopus ID

105041603917

Recommended Citation

Mohamed, Walid; Abdelhalim, Ibrahim; Hassan, Tayseer; Abdelaziz, Amr; Badawy, Mohammed; El-Melegy, Moumen; Ali, Asem; Mahmoud, Ali; Taher, Fatma; Ghazal, Mohammed A.; Contractor, Sohail; and El-Baz, Ayman, "A Multimodal Deep Learning Framework for Prostate Cancer Risk Prediction Integrating Biparametric MRI, Radiology Reports, and Clinical Biomarkers" (2026). All Works. 8201.
https://zuscholars.zu.ac.ae/works/8201

Indexed in Scopus

yes

Open Access

no