Negative effects of unlabeled response scales

Lars Bergkvist, Zayed University

ORCID Identifiers

0000-0002-4271-9182

Document Type

Article

Source of Publication

International Journal of Market Research

Publication Date

12-23-2020

Abstract

This study used a novel research approach to investigate the effects of unlabeled response scales on response distributions. Instead of responding to standard questionnaire items respondents were asked to report given judgments on either semantic-differential (SD) or agree-disagree (AD) response scales, thereby showing the extent to which respondents agree upon where to place given judgments. Results from a survey-based study ( N = 418) show that respondents to a large extent disagree about where to place judgments on the response scale; the level of agreement for different judgment intensities ranged from 42% to 82% and the level of agreement is lower for AD than SD response scales. The low levels of agreement contribute to non-substantive variance in the data which increases the risk of attenuated or inflated correlations between constructs. Moreover, simulations of actual response distributions suggest that unlabeled response scales may lead to a strong bias in the form of underestimated shares of positive answers. Implications for research and marketing research practice of using unlabeled response scales are discussed and it is recommended that response categories on SD and AD items always should be labeled since this will reduce non-substantive variance and bias in the data.

ISSN

1470-7853

Publisher

SAGE Publications

First Page

147000000000000

Disciplines

Business | Computer Sciences

Scopus ID

85120428240

Indexed in Scopus

yes

Open Access

no

Link to Full Text

Share

COinS