Evaluating ChatGPT-5 for Misuse Case Diagram Generation: An Empirical Evaluation

Document Type

Conference Proceeding

Source of Publication

International Conference on Evaluation of Novel Approaches to Software Engineering Enase Proceedings

Publication Date

1-1-2026

Abstract

Misuse case diagrams are a widely adopted technique in security requirements engineering, enabling analysts to model adversarial threats and derive countermeasures early in the software development lifecycle. However, manual construction of these diagrams is prone to incompleteness and subjectivity, requiring significant security expertise. Large language models (LLMs) such as ChatGPT present a promising opportunity to automate this process, yet their effectiveness for generating structured security modeling artifacts remains largely unexplored. This paper presents an exploratory study evaluating ChatGPT-5's ability to generate misuse case diagrams directly from textual security requirements, using 12 case studies of varying complexity spanning small, medium, and large requirement sets. The diagrams produced by ChatGPT-5 were evaluated against manually constructed ground-truth diagrams, and our results indicate that ChatGPT-5 performs well overall, demonstrating a strong capability to identify key actors, threats, and adversarial relationships from natural language input.

ISBN

[9789897588280]

ISSN

2184-4895

Publisher

SCITEPRESS - Science and Technology Publications

Volume

1

First Page

599

Last Page

610

Disciplines

Computer Sciences

Keywords

ChatGPT-5, Empirical Evaluation, Large Language Models, Misuse Case Diagrams, Security Requirements Engineering, Threat Modeling, UML

Scopus ID

105046597118

Indexed in Scopus

yes

Open Access

yes

Open Access Type

Gold: This publication is openly available in an open access journal/series

This document is currently not available here.

Share

COinS