"Contrastive concept-phrase pre-training for generating clinically accu" by Abdallah Tubaishat, Tehseen Zia et al.
 

Contrastive concept-phrase pre-training for generating clinically accurate and interpretable chest X-ray reports

Document Type

Article

Source of Publication

Neural Computing and Applications

Publication Date

1-1-2024

Abstract

Automated radiology report generation is an emerging field for improving patient care and alleviating radiologist workload. However, existing methods face a range of challenges such as limited data availability, clinical metric performance, and interpretability. To address these issues, we propose a contrastive concept-phrase pre-training (C2P2) method, which utilizes a phrase-concept grounding task for contrastive learning. C2P2 learns the correspondence between phrases in a report and image concepts by using a phrase classification task to train a multi-label classifier for X-rays and extracting visual concepts of phrases using class activation maps. We then fine-tune a pre-trained BERT model to translate the extracted phrases into reports. Our proposed method outperforms or matches the previous state of the art in clinical efficacy metrics on both internal and external datasets. Moreover, C2P2 leverages more vision language data for pre-training and provides visual explanations of generated phrases.

ISSN

0941-0643

Publisher

Springer Science and Business Media LLC

Disciplines

Computer Sciences

Keywords

Chest X rays, Contrastive learning, Language image pre-training, Medical reports

Scopus ID

85211611230

Indexed in Scopus

yes

Open Access

no

Share

COinS