The need for guidelines
The use of international concepts and measurement methodology is fundamental to official statistics. Such standards contribute to quality by ensuring that best practice is followed internationally and that official statistics are internationally comparable. This is reflected in the UN Fundamental Principles of Official Statistics. In particular, principle 9 states that:
The use by statistical agencies in each country of international concepts, classifications and
methods promotes the consistency and efficiency of statistical systems at all official levels.
Although measures of subjective well-being are now available from both official and non-official sources for an increasing range of countries, these measures currently lack commonly accepted international guidelines for their collection and dissemination. The only measures of subjective well-being on which cross-country comparisons can currently be made on a consistent basis are derived from non-official sources, and these face limitations associated with relatively small sample sizes, limitations of sample design and low survey response rates (Box 1).
Box 1. Non-official sources of subjective well-being data
Measures of subjective well-being are not currently collected in a systematic and consistent way across OECD national statistical agencies. While a number of OECD countries do collect measures of subjective well-being as part of their official statistics, and in some cases have been doing so for some time, official measures currently lack the consistency needed for them to be used as the basis for international comparisons. There are, however, a number of datasets currently available that contain measures of subjective well-being covering a wide range of countries. Indeed, much of the current body of knowledge regarding the validity and properties of measures of subjective well-being is derived from the analysis of these non-official datasets.
The two largest datasets containing comparable measures of subjective well-being are the Gallup World Poll and the World Values Survey. The Gallup World Poll started in 2005, and now covers 132 countries; the sample size is about 1 000 respondents per country per wave, with plans to increase such sample size to 4 000 respondents in all countries with a population of 25 million and over by 2012. The Gallup World Poll is an annual survey and includes measures of life evaluation and a range of questions related to current mood and emotional experiences (affect). The World Values Survey has a longer history (although with uneven sampling quality), with the first wave having been collected between 1981 and 1983 and covering 15 countries. There have been four subsequent waves, with the most recent wave collected between 2005 and 2008 and covering 56 countries. A sixth wave is currently being collected (2011-12). The World Values Survey contains measures of life evaluation and overall happiness, as well as more focused measures of experienced mood and aspects of psychological well-being in the more recent waves.
While the Gallup World Poll and the World Values Survey are usually taken as providing a reference point for questions on subjective well-being across countries, there are a number of additional surveys that complement these in various ways. The European Social Survey provides information on a number of aspects of subjective well-being for a varying range of European countries between 2002 and 2010. In the 2006 wave of the European Social Survey, a module was included to collect detailed information on the “eudaimonic” aspects of well-being (i.e. meaning, purpose, flourishing), thus expanding the range of subjective well-being concepts measured beyond evaluations and affect. A repeat of this module will be carried out in 2012. In addition, the triennial European Quality of Life Survey contains extensive information on subjective well-being.
Eurobarometer is a regular opinion survey covering European Union nations that has been collected since 1973. Although the subjective well-being questions contained in Eurobarometer are relatively limited, they provide the longest unbroken time series for measures of subjective well-being for a cross-section of countries. Similar question have also been included in several waves of the Latinobarometro.
In addition to these cross-sectional surveys, a number of panel surveys have been widely used by researchers to analyse subjective well-being. In particular, the German Socio-Economic Panel and the British Household Panel Study are high-quality panel surveys that include information on subjective well-being. The German Socio-Economic Panel dates back to 1984 and runs to the present day, with a total sample of over 12 000 households. By comparison, the British Household Panel Study dates back only to 1991, but has recently been integrated into the UK Household Longitudinal Study (also known as “Understanding Society”), with a total sample of over 40 000 households. Because both of these studies follow the same person through time, they have been crucial in allowing researchers to understand the interaction between unobserved personality traits, life-events, environmental changes (including policy changes) and responses to subjective well-being questions.
Although non-official data sources have provided much information on subjective well-being, they do have several distinct limitations. With the exception of the large panel studies, most non-official data sources have relatively small sample sizes that limit the conclusions which can be reached about changes in levels of subjective well-being and differences between groups. Many of the main non-official surveys also are affected by low response rates and have sample frames that are not as representative as is the case for official surveys. Finally, the developers of non-official surveys often have fewer resources available for cognitive testing and survey development than is the case for national statistical offices. Thus, although existing non-official data sources have provided a great deal of information on subjective well-being, there remain a range of questions that will not be answered until high-quality large-scale official surveys are available.
While the academic literature contains extensive information about which subjective well-being measures to collect and how to collect them, no consistent set of guidelines currently exist for national statistical agencies that wish to draw on this research. For official measures of subjective well-being to be useful as indicators of national progress, these official measures should be collected in a consistent manner, which, in turn, requires an agreed way to collect such measures. This drives the need for developing commonly accepted guidelines around the measurement of subjective well-being, even if such guidelines will need to be revised in the future as more information becomes available on subjective well-being.
Guidelines are also needed because subjective well-being measures are strongly affected by question structure and context, and the results from differently worded questions (or even a different ordering of similar questions) are likely to affect comparability. Yet comparability is a key point of interest for decision-makers, who will often want to benchmark the progress of one region, country or population group against another. While interpreting such comparisons can be difficult due to issues such as cultural biases in response styles, consistency in measurement can eliminate other potential sources of bias. It is important that, where there are differences in measured levels of subjective well-being, these are not falsely attributed some significance when, in fact, the difference actually reflects the impact of question wording or context.