Towards Automatic Narrative Coherence Prediction

Document Type

Conference Proceeding

Source of Publication

2021 International Conference on Multimodal Interaction (ICMI '21)

Publication Date



Research in Psychology has shown that stories people tell about themselves, and how they recall their experiences, reveal a lot about their individual characteristics and mental well-being. The Narrative Coherence Coding Scheme (NaCCS) is a set of guidelines established in psychology research for annotating the “coherence” of a narrative along three dimensions: context, chronology and theme. A significant correlation was found between a narrative’s coherence score and independently collected mental health markers of the narrator. Currently, all coherence annotations are done manually; a time consuming task which drains vital resources. In this paper, we propose an Artificial Intelligence based approach involving Natural Language Processing (NLP) to predict a narrative’s coherence score (4-class classification problem). We explore a number of techniques, ranging from traditional machine learning models such as Support Vector Machines (SVM) to pre-trained language models such as BERT (Bidirectional Encoder Representations from Transformers). BERT produced the best results for all dimensions in terms of accuracy: 53.7% (context), 71.8% (chronology), and 69.6% (theme). The location of information in the narratives (beginning, end, throughout) was helpful in improving predictions.


Association for Computing Machinery (ACM)


Computer Sciences


NLP, AI for mental health, Machine Learning (ML), Narrative text analysis, Narrative Coherence Coding Scheme (NaCCS), (Un)supervised learning, Word embedding, BERT

Indexed in Scopus


Open Access