Large language models in NLP: evolution, architectural trends, and open challenges

Document Type

Article

Source of Publication

Journal of Big Data

Publication Date

12-1-2026

Abstract

The rise of Large Language Models (LLMs) has transformed how Natural Language Processing (NLP) and its subdomains are approached. Recent technological advancements have driven this transformation. This study offers researchers a detailed overview of LLMs, comparing them with traditional rule-based systems, statistical techniques, machine learning, neural networks, and the rise of transformer-based architectures. From a wider perspective, language models such as GPT, BERT, T5, PaLM, and LLaMA have facilitated the transformation of entire sectors, including healthcare and business, due to their highly scalable nature. Despite their wide range of applications, LLMs face numerous challenges, such as output biases, limited interpretability, high computational requirements, low granularity, and, most importantly, limited accessibility. This paper highlights modern adaptive learning processes, domain-specific applications, and multimodal learning, along with critical issues such as fairness, ethical deployment, and sustainability. There is a growing need for more effective regulation, the development of multilingual capabilities, and, above all, the creation of general-purpose AI systems to enhance inclusion. This research emphasizes ethical accountability by examining the social implications of advanced AI systems. It further promotes interdisciplinary collaboration to develop AI systems that are ethical, effective, and socially responsible within a clearly defined decision-making framework for research and application.

ISSN

2196-1115

Publisher

Springer Science and Business Media LLC

Volume

13

Issue

1

Disciplines

Computer Sciences

Keywords

BERT, Comparative analysis, GPT, Large language models, Natural language processing, Transformers

Scopus ID

105042336817

Indexed in Scopus

yes

Open Access

yes

Open Access Type

Gold: This publication is openly available in an open access journal/series

This document is currently not available here.

Share

COinS