Grammaticality judgment task: Reliability and scope

Muhammad Asif Qureshi, Zayed University

Document Type

Article

Source of Publication

Journal of Asia TEFL

Publication Date

6-1-2020

Abstract

© 2004 AsiaTEFL.org. All rights reserved. Although extensively used, concerns have been expressed about the reliability and generalizability of grammaticality judgment tasks (GJTs; Alanazi, 2015). It has been argued that learners might guess grammaticality based on feel (Bialystock, 1979) and that the GJT-based results ignore grammatical complexity (Ellis, 1991). While several studies have attempted to validate the results of a GJT against other tasks, the tasks used in these studies required language production (e.g., Leow, 1996; Mandell, 1999), which is different from error identification and correction. The current study used an editing task (ET) to validate learners’ performance on a GJT. An editing task, like a GJT, requires identification of grammatical inaccuracies. Besides, it situates errors in a meaningful context and offers opportunities for multiple corrections of the same error. Overall, 311 participants took part in the study. A paired sample t-test showed a significant difference (t = 24.10, p = .00, d = 1.91) between the two tasks. Fifteen percent of the data (45 participants) was further inspected to have a better understanding of learners’ error correction-pattern on the editing task. The results revealed that about 20% of the errors identified were wrongly corrected, which exposes limitations of the GJTs that only require judging the well-formedness of a construction. Moreover, performance on the editing task revealed a greater level of diversity in learners’ responses. On average, eight of the total twelve grammatical features were corrected in 5.87 (SD = 2.47) different ways. The variety of responses on the ET reveals the dynamic nature of learners’ interlanguage that allows for multiple ways for correction. The ET, as compared to the GJT, provides an ecologically more valid way to assess L2 learners’ grammatical proficiency.

ISSN

1738-3102

Publisher

Asian Association of Teachers of English as a Foreign Language

Volume

17

Issue

2

First Page

349

Last Page

362

Disciplines

Education

Keywords

Grammaticality judgment tasks, Reliability, Scope

Scopus ID

85089705553

Indexed in Scopus

yes

Open Access

yes

Open Access Type

Gold: This publication is openly available in an open access journal/series

Link to Full Text

Share

COinS