Fine-Tuned Organization-Specific LLMs for Security Compliance: Improving Accuracy and Style Consistency

Document Type

Conference Proceeding

Source of Publication

2026 IEEE International Conference on Smart Sustainable Systems for Computer and Engineering Applications 3scea 2026

Publication Date

4-19-2026

Abstract

Organizations are required to maintain security documentation aligned with standards such as ISO/IEC 27001, NIST SP 800-53, and UAE IA. Despite this requirement, many continue to rely on manual and error-prone processes. Recent studies have investigated the application of large language models (LLMs) in compliance tasks. However, most efforts on fine-tuning and domain-specific models have focused on control mapping, requirement alignment, and policy validation, rather than generating complete security documentation. Generalpurpose LLMs remain limited by issues of accuracy, formatting, and confidentiality. This work looks at how a fine-tuned language models (LLMs) trained on organization-specific material can be used to generate security documentation. The generated documents are then compared with those produced by a general-purpose model, focusing on compliance requirements, terminology, and consistency of writing. Evaluation is performed using a compliance checklist, an automated style checker, and GRC auditors' reviews. The proposed evaluation framework helps determine the extent to which customized large language models (LLMs) can support the preparation of compliant documentation and be used in real organizational compliance processes in a practical and secure manner.

ISBN

[9798331556686]

Publisher

IEEE

First Page

49

Last Page

54

Disciplines

Computer Sciences

Keywords

compliance accuracy, compliance automation, fine-tuning, general-purpose LLMs, ISO/IEC 27001, large language models, organization-specific LLMs, security documentation

Scopus ID

105046110774

Indexed in Scopus

yes

Open Access

no

Link to Full Text

Share

COinS