Towards Automatic Narrative Coherence Prediction

Document Type

Conference Proceeding

Source of Publication

2021 International Conference on Multimodal Interaction (ICMI '21)

Publication Date

10-18-2021

Abstract

Research in Psychology has shown that stories people tell about themselves, and how they recall their experiences, reveal a lot about their individual characteristics and mental well-being. The Narrative Coherence Coding Scheme (NaCCS) is a set of guidelines established in psychology research for annotating the “coherence” of a narrative along three dimensions: context, chronology and theme. A significant correlation was found between a narrative’s coherence score and independently collected mental health markers of the narrator. Currently, all coherence annotations are done manually; a time consuming task which drains vital resources. In this paper, we propose an Artificial Intelligence based approach involving Natural Language Processing (NLP) to predict a narrative’s coherence score (4-class classification problem). We explore a number of techniques, ranging from traditional machine learning models such as Support Vector Machines (SVM) to pre-trained language models such as BERT (Bidirectional Encoder Representations from Transformers). BERT produced the best results for all dimensions in terms of accuracy: 53.7% (context), 71.8% (chronology), and 69.6% (theme). The location of information in the narratives (beginning, end, throughout) was helpful in improving predictions.

Publisher

Association for Computing Machinery (ACM)

Disciplines

Computer Sciences

Keywords

NLP, AI for mental health, Machine Learning (ML), Narrative text analysis, Narrative Coherence Coding Scheme (NaCCS), (Un)supervised learning, Word embedding, BERT

Scopus ID

85118970490

Indexed in Scopus

yes

Open Access

no

Link to Full Text

Share

COinS