Response styles and the cultural context Introduction

Response biases and heuristics have been cross-cutting themes throughout this chapter, with an emphasis on how survey methodology can affect their likelihood. As noted in the introduction, however, the risk of response biases, heuristics and error is essentially the product of a complex interaction between methodological factors (such as the cognitive demands made by certain questions), respondent factors (such as motivation, fatigue and memory) and the construct of interest itself (such as how interesting or relevant respondents find it).

Where present, response biases can affect the accuracy of self-report survey data. The precise nature of the effect depends, however, on the bias in question, what the bias has been caused by, and whether it is affecting all respondents and all items in a survey similarly and consistently over time. Some of these various possible biases, sources and impacts are summarised and illustrated in Table 2.3. Key risks highlighted include increased error in the data, biased mean scores (up or down), and risks to the overall accuracy of comparisons between groups, over time or between different surveys.

Table 2.3. Illustrative examples of response biases and their possible effects

Source of response bias or heuristic

Type of bias observed

Potential respondents affected

Potential impact on responses

Potential risks to further analyses

Discussed in

the present chapter in...

Question wording or response format encourages acquiescence

Acquiescence.

Potentially all, but some may be more susceptible to acquiescence than others (e.g. those with lower motivation).

Responses biased in the direction of the positive response category.

  • • Risk of inflated associations with any other variables that use the same response scale.
  • • Risk of less accurate comparisons between groups if some groups are consistently more susceptible than others.

Sections 1 and 2.

Prior survey questions prime respondents to think about certain information when responding

Priming (question order effects).

Potentially all, but some may be more susceptible to context effects than others (e.g. those with lower motivation).

Responses biased

in the direction of the prime.

  • • Risk of less accurate comparisons between surveys with different question ordering.
  • • Risk of inflated associations between the variable

and the prime.

• Risk of less accurate comparisons between groups

if some groups are consistently more susceptible to context effects than others.

Section 3.

Sample includes some fatigued or unmotivated respondents

Satisficing (respondents more likely to exhibit response biases or use heuristics).

Only those experiencing

fatigue/lower

motivation.

Various, depending on the response bias or heuristic used by respondent.

  • • Random (largely unpredictable) error introduced.
  • • Risk of less accurate comparisons between surveys (e.g. if respondents less fatigued by other survey methods).
  • • Risk of less accurate comparisons over time

if respondents less fatigued on subsequent occasion.

Sections 1, 2, 3 and 4.

Linguistic or cultural response “styles” (e.g. towards more moderate or more extreme responding)

Moderate responding/ extreme responding.

Different respondents affected in different ways (depending on language/culture).

Responses biased towards centre of response scale (for moderate responding) or towards extremes of response scale (for extreme responding).

  • • Risk of less accurate comparisons between groups, based on language or culture.
  • • Minimal risk to comparisons overtime and between surveys.

Section 5.

However, not all types of bias are a problem for all types of analyses, thus the management of response bias depends on both the nature of the bias and the nature of the analysis being performed.

Sections 1 and 2 of the current chapter explored the ways in which question wording and response formats can contribute to communication, motivation and memory failures - each of which are thought to present risks to the quality of subjective well-being data by making response biases and the reliance on response heuristics more likely. Section 3 meanwhile considered the extent to which priming and question context effects can influence patterns among subjective well-being data. Finally, in the course of examining mode effects, Section 4 discussed the extent to which subjective well-being measures may be affected by socially desirable responding.

In addition to the various methodological features that can influence response biases, it has been suggested that respondents themselves can also exhibit consistent response styles - i.e. a repeated tendency to rely on a particular response heuristic or show a relatively constant susceptibility to certain forms of response bias. This final section of the chapter explores the evidence in relation to response styles and subjective well-being in general, and the latter half of the section focuses in particular on the risk of cultural response styles that might affect the comparability of data between countries.

 
Source
< Prev   CONTENTS   Source   Next >