From Image to Insight: Evaluating LLM Accuracy in Understanding UML Use Case Diagrams with Claude

Author First name, Last name, Institution

• Mohamed El-Attar, Zayed University, Abu Dhabi CampusFollow
• Yasser Khan, FAU College of Engineering and Computer Science
• Mahmood Niazi, King Fahd University of Petroleum and Minerals
• Sajjad Mahmood, King Fahd University of Petroleum and Minerals
• Mohammad Alshayeb, King Fahd University of Petroleum and Minerals

Document Type

Conference Proceeding

Source of Publication

International Conference on Evaluation of Novel Approaches to Software Engineering Enase Proceedings

Publication Date

1-1-2026

Abstract

UML use case diagrams are a prominent artefact of requirements engineering, capturing the functional scope of a software system in terms of actors, use cases, and their stereotyped relationships. The emergence of multimodal large language models with image understanding capabilities raises the question of whether such models can reliably extract structured construct-level information from use case diagram images. This paper reports an empirical evaluation of Claude on the task of counting 14 notational construct types from a corpus of 78 computer-generated UML use case diagrams, assessed against manually verified ground truth annotations. Results reveal a strongly differentiated accuracy profile: Claude achieved near-perfect exact match rates (>=95%) for visually unambiguous constructs such as Actors, System Boundary, Extend, and Extension Points, but performed poorly on association directionality. Diagram structural complexity was the strongest predictor of overall error. The findings indicate that LLM-based use case diagram analysis is construct-selective and complexity-sensitive and provide empirically grounded guidance for practitioners considering LLMs as tools for automated diagram annotation in software maintenance workflows.

DOI Link

10.5220/0015037500004015

ISBN

[9789897588280]

ISSN

2184-4895

Publisher

SCITEPRESS - Science and Technology Publications

Volume

1

First Page

631

Last Page

641

Disciplines

Computer Sciences

Keywords

Claude, Construct-Level Accuracy, Diagram Interpretation, Empirical Software Engineering, Image-Based Model Extraction, Large Language Models, Model-Driven Engineering, Multimodal AI, Requirements Engineering, Software Documentation, Software Maintenance, UML Use Case Diagrams, Visual Diagram Understanding

Scopus ID

105046611055

Creative Commons License

This work is licensed under a Creative Commons Attribution-NonCommercial-No Derivative Works 4.0 International License.

Recommended Citation

El-Attar, Mohamed; Khan, Yasser; Niazi, Mahmood; Mahmood, Sajjad; and Alshayeb, Mohammad, "From Image to Insight: Evaluating LLM Accuracy in Understanding UML Use Case Diagrams with Claude" (2026). All Works. 8331.
https://zuscholars.zu.ac.ae/works/8331

Indexed in Scopus

yes

Open Access

yes

Open Access Type

Gold: This publication is openly available in an open access journal/series