How can the impacts of different drivers be compared?
Particularly in the case of investigating policy trade-offs, or when deciding between two different courses of action, there may be times when it is useful to focus on the relative size of different drivers of subjective well-being. Directly comparing the regression coefficients associated with different drivers requires caution, however. Challenges include the need to consider units of measurement as well as the potential for correlation among the drivers, and problems potentially arising due to shared method variance or self-report biases. The generalisability of findings obtained through regression analysis is also of crucial importance for understanding policy implications.
Although the issues associated with comparison of regression coefficients apply to all regression-based analyses, they are particularly relevant here because there are often strong inter-correlations among the drivers of subjective well-being, and because a growing literature suggests some degree of reverse-causality between life evaluations and its drivers in particular. The problems of shared method variance and the possibility of self-report “bias” are also discussed. Finally, a key consideration for policy evaluations in particular are the time horizons over which well-being drivers might be expected to take effect.
Regression coefficients and correlations among independent variables. As noted earlier, when there are no interrelationships between independent variables, the size of the regression coefficient gives an indication of how a one unit change in the independent variable can be expected to influence the dependent variable. However, because the high-level drivers of subjective well-being (such as income, health status and social connections) are likely to be so strongly interrelated, interpretation of their individual contributions must proceed with caution, because there may be mediation, confounding and suppression effects in the data.
Although conceptually distinct, mediation, confounding and suppression each describe scenarios when relationships between an independent variable and a dependent variable are affected by the presence of a third related independent variable (the mediator, confound or suppressor). When the third variable is actually measured, these effects can be detected by a substantive change in the regression coefficient for the independent variable when the third variable is included in the model (relative to a model that excludes the third variable). In the case of mediation, the third variable is described as “transmitting” the effect of the independent variable to the dependent variable. In the case of confounding effects, the third variable is described as a “nuisance” variable, producing spurious correlations between the independent variable and the dependent variable.36 Conversely, when suppression effects are present, relationships between the independent and the dependent variable become stronger when the third variable is included in the model. In the event that the third variable remains unmeasured, the coefficient observed for the independent variable can be misleading (see Dolan, Peasgood and White, 2007).
Boarini et al. (2012) also raise the possibility of “over-measurement” of individual drivers - where, if several measures of the same driver are included, correlations among the measures can crowd one another out, such that some relevant variables fail to reach significance in the overall model. This means that a significant driver could be overlooked if there are too many measures of it in the model, because the overall effect will be distributed among too many independent variables.
The fact that the regression coefficient for an independent variable is often dependent on the other variables in the regression equation means that selecting the variables to include in an analysis is a crucially important task. A clear theoretical structure and an understanding of the hypothesised causal pathways must underpin these decisions. While the use of hierarchical (or sequential) regression and structural equation modeling can provide an analytical strategy for examining causal pathways among variables, techniques of this sort cannot provide a definitive solution with regard to the relative “importance” of interrelated independent variables, in terms of absolute size of impact.
The omitted variable problem. In addition to the mediating, confounding and suppression that can occur as a result of measured variables, causal inferences about relationships between variables can be severely hampered by unmeasured or “omitted” variables. Specifically, a significant statistical relationship can be observed between two variables not because there is a causal relationship between them, but because both variables are causally related to a third unobserved variable that has been omitted from the analyses.37 Omitted variables can also suppress causal relationships between observed variables, causing results to fail to reach statistical significance due to unmeasured factors.38 This is a problem right across econometric analyses (and all tests of association) and is in no way limited to examination of the drivers of subjective well-being.
Among subjective well-being data sets, because so many drivers help to explain final subjective well-being outcomes, several counter-intuitive findings (such as repeated failures to find relationships between income growth and subjective well-being, despite strong cross-sectional relationships between income and well-being) could potentially be explained with reference to variables that have been omitted from analyses (such as changes in relative income, or patterns of decline in other important determinants of subjective well-being, such as health, social connections, perceived freedom, corruption, etc.). The effects of relative income, and aspirations about income, have in particular been studied by several authors (for reviews, see Dolan, Peasgood and White, 2007, and Clark, Frijters and Shields, 2008) and reflect the frame-of-reference effects discussed in Section 1 of this chapter.
Another set of omitted variables often discussed in relation to subjective well-being involve personality- and temperament-based measures. Individual fixed effects do appear to account for a sizeable proportion of the variance in subjective well-being measures (see Diener, Inglehart and Tay, 2012), and some of these fixed effects may in turn reflect dispositional tendencies. For example, Lucas and Donnellan (2011) reported that 34-38% of variance in life satisfaction was due to stable trait-like differences - although this study did not include measures of the objective life circumstances that might impact on stable trait-like components. The issue of whether these stable differences reflect a true causal impact of personality and temperament on experienced subjective well-being, or simply a response style that affects self-reported measures (including subjective well-being, but also health and exposure to stress) is discussed in relation to shared method variance, below.
Self-report measures and shared method variance. A final factor to consider in the interpretation of subjective well-being drivers is shared method variance,39 also known as common method variance, which can inflate the estimated impact of self-reported drivers relative to those measured through other means (such as objective observations). In particular, due to a combination of social desirability biases, response sets, differences in scale interpretation or use, and similarities between the questions themselves, one might expect that subjective well-being and other self-report measures such as self-rated health, self-reported mental health, self-reported social connections, and/or personality and dispositional variables might have correlated errors. Indeed, response formats to such questions are often very similar (e.g. 0-10 labelled scales). Furthermore, several items on current measures of eudaimonia and affect bear a strong resemblance to some of the questions used to measure personality and mental health. Concepts such as self-efficacy, often included in constructions of eudaimonia, are also often considered to be an aspect of personality or dispositional tendency.
When comparing the effects of different drivers of subjective well-being, particularly in cross-sectional data, it is therefore important to consider how each of those determinants was measured. The possibility of shared method variance has led some authors to suggest that dispositional measures such as personality or negative affectivity40 should be included as control variables in analyses of self-report data, and particularly when analyses are cross-sectional (Brief et al., 1988; Burke, Brief and George, 1993; McCrae, 1990; Schaubroeck, Ganster and Fox, 1992), in order to remove any bias associated with subjective self-report processes more generally. The risk in doing so, however, is that this could potentially swamp the effects of other important determinants, and remove variance in subjective well-being data that is likely to be of policy interest. For example, if exposure to childhood poverty or long-term health problems influences responses to both personality and subjective well-being questions, controlling for personality in the analyses could mask the true impact of childhood poverty and/or long-term health problems on the outcomes of interest. Personality, and negative affectivity in particular, may also play a substantive role in the overall development of subjective well-being (Moyle, 1995; Bolger and Zuckerman, 1995; Spector et al., 2000).
An alternative approach to investigating the issue of shared method variance and self-report bias is to use longitudinal panel data, in which individual fixed effects can be controlled. In such models, the ability of self-reported drivers to predict changes in subjective well-being over time can be investigated, and this is a much stronger test of causality. These types of analyses enable the effects of more objective indicators to rise to the forefront, whilst problems associated with shared method variance recede into the background.
Reverse and two-way causality. Understanding the direction of causality when examining drivers of subjective well-being is crucial to establishing their policy-relevance. As noted in the data requirements section above, an analyst’s ability to make causal inferences is strongest where experimental data, or data from randomised controlled trials, is available. Quasi-experimental designs and longitudinal panel data can also offer insights into likely causal relationships, because analyses can be restricted to factors that temporally precede changes in subjective well-being over time. In cross-sectional data, the ability to make causal inferences is severely limited - and thus results need to be interpreted alongside evidence about the direction of causality from other sources.
In regression-based analyses, one method for exploring issues of reverse-causality is to include an instrumental variable.41 Instrumental variables are sometimes used when there are problems of endogeneity in regression models - i.e. when the independent variable of interest is correlated with the model error term. Two-way or reverse causality can be a key source of endogeneity, as can omitted variables (described above). Dolan and Metcalfe (2008) and Powdthavee (2010) report using instrumental variables to obtain better estimates of the exogenous effect of income on life evaluations. This typically increases the estimate of the income coefficient (Fujiwara and Campbell, 2011). In practice, however, it is very difficult to identify appropriate instrumental variables for income, as most of the key variables strongly associated with income also tend to be associated with life satisfaction.
Generalisability of results. Analyses of drivers are strongly affected by both the variables included in the model and the population sampled - which in turn both influence the extent to which results can be generalised. The importance of different drivers of subjective well-being may vary systematically according to certain group characteristics, because different groups within and across societies may be characterised by very distinct initial resource endowments. For example, Boarini et al. (2012) examined the determinants of life satisfaction among different population sub-groups (i.e. by gender, age and participation in the labour market) across 34 OECD countries. While the overall pattern of coefficients was quite similar, there were a number of non-trivial differences in the subjective well-being functions42 observed in the different groups.
This evidence suggests that, for different population sub-groups, the relative impact of the determinants of subjective well-being may differ. Heterogeneity in the relative size and significance of the drivers of subjective well-being has implications for how we might inform the public about the relative importance of the different drivers. Policies aimed at increasing subjective well-being may also need to consider the distribution of well-being resource endowments among different population sub-groups. Regression analyses generate results for the average individual - and in practice, there may be wide individual differences in the specifics of the well-being function. Different people may find happiness in different ways.
Although several studies have highlighted strong consistencies among affluent countries in terms of the direction and significance of some of the high-level determinants of subjective well-being (Helliwell and Barrington-Leigh, 2010; Fleche, Smith and Sorsa, 2011), one might also expect to see some differences in subjective well-being functions between countries, because countries also vary in terms of both their initial resource endowments and how those resources are distributed. For example, Inglehart et al. (2008) found that among less economically-developed countries, there were stronger associations between happiness and in-group solidarity, religiosity and national pride, whereas at higher levels of economic security, free choice becomes a more important predictor. Drawing on data from the Gallup World Poll, Bjprnskov (2010) reports that Cantril Ladder life evaluations showed a strong relationship with levels of GDP per capita among countries with lower relative incomes, whereas social trust became a strong and significant determinant only among countries with higher relative incomes. In the same vein, Helliwell and Barrington-Leigh (2010) show that coefficients on a number of social variables are higher in OECD than in non-OECD countries, while the coefficients on log income were identical in the two parts of the global sample.