Measurement error in estimating regression coefficients
Monetary valuations obtained used subjective well-being data are typically based on regression coefficients, and thus require a high degree of precision in estimating those coefficients. Measurement error among the set of drivers (independent variables) examined in the course of valuations can be especially problematic. Of particular concern is the measurement error in self-reported income - which risks reducing the income coefficient, leading to higher valuations of non-market factors. For example, Powdthavee (2009) found an increased income coefficient (producing lower valuations for non-market factors) where objective income information was obtained by interviewers through examination of payslips.
Dolan and Metcalfe (2008) and Powdthavee (2010) also report using instrumental variables46 in subjective well-being valuations to obtain better estimates of the exogenous effect of income on life evaluations. Fujiwara and Campbell (2011) note that instrumenting for income typically increases the estimate of the income coefficient, thus producing lower overall valuation estimates for non-market factors. For example, in Dolan and Metcalfe’s analysis, this correction brings estimates of the value of urban regeneration down from GBP 19 000 to around GBP 7 000. Powdthavee reports that this technique lowers the valuations of marriage from around GBP 200 000 to GBP 3 500 per annum. In practice, however, it is very difficult to identify appropriate instrumental variables for the purposes of valuations, as most of the key variables strongly associated with income also tend to be associated with life satisfaction.
Measurement error in non-market factors could also reduce coefficients attached to the variable in question, leading to under-valuation. Conversely, if measurement error in non-market factors is positively correlated with measurement error in the life evaluations (for example, due to shared method variance or response biases), this could inflate rather than depress their coefficients, leading to over-valuation, unless the income variable was similarly affected. Again, instrumental variables could be of particular use in separating out causal effects from correlated errors.
The large impact that measurement error in independent variables can have on valuations means that, particularly when small or non-representative samples are used in regressions, it will be essential to check the coefficients obtained for income and other variables in the model (and especially for the non-market factor in question) to ensure that they fall within the range that might be expected, based on larger and more representative samples - and preferably those utilising high-quality panel data (Box 4.7). Further work on potential instrumental variables for use in valuations will be important for future development of the technique.
Box 4.7. The range of income estimates observed in the life satisfaction literature
The method used to estimate the value of non-market factors on the basis of life satisfaction data is very sensitive to the coefficient estimated for income. When interpreting the results of such valuations it is therefore helpful to consider the range of coefficients for income identified in the wider literature.
A very large number of authors have examined the role of income in life evaluations and the issue of whether increases in a country's average income over time are associated with increases in a country's subjective well-being (e.g. Easterlin, 1974, 1995; 2005; Hagerty and Veenhoven, 2003; Sacks, Stevenson and Wolfers, 2010). In practice, the effects of income are highly complex, and can vary both between countries and within different population sub-groups. Some authors report a consistent finding that income plays a more important role in developing and transition countries, and a less important role in more affluent societies (Bj0rnskov, 2010; Clark, Frijters and Shields, 2008; Sacks, Stevenson and Wolfers, 2010), whereas others report a similar magnitude effect for income across all countries (Deaton, 2008; Helliwell, 2008; Helliwell and Barrington-Leigh, 2010). Estimates for income coefficients are also critically sensitive to other variables included in the regression model. Clark, Frijters and Shields (2008), Sacks, Stevenson and Wolfers (2010), and Fujiwara and Campbell (2011) provide a more extensive overview of several other important issues - including those associated with reverse causation, individual effects and the importance of relative income (i.e. an individual's income in comparison to a given reference group). A final issue is the problem of income non-response rates, which are rarely reported but could also affect coefficients estimated for income.*
Although the results from any one study should be interpreted with caution, research described below illustrates how estimates for the effect of income can vary when different background characteristics and life circumstances are controlled. In all cases, a 0-10 life evaluation measure and log-transformed income data are used.
- • Sacks, Stevenson and Wolfers (2010) report results from several large life evaluation data sets, together spanning 140 countries. In cross-sectional data (pooled across all countries) and controlling only for country fixed effects, regression coefficients for log household income on Cantril Ladder life evaluations range from 0.22 to 0.28. Results remain similar when controlling for age and sex, while adjusting for the effects of permanent income or instrumenting income increases coefficients to between 0.26 and 0.5. The authors conclude that at the within-country level, the coefficient for the permanent effect of income lies somewhere between 0.3 and 0.5. They also suggest similar magnitude effects at the between-country level and for changes in income over time.
- • Boarini et al. (2012) use two waves of Gallup World Poll data (2009 and 2010) to examine the determinants of Cantril Ladder life evaluations among 34 OECD countries. Pooled across countries, the coefficient for log household income is estimated at 0.18 when only key background characteristics are controlled. Controlling for a variety of other individual-level well-being outcomes (health problems, social connections, environmental quality, personal security, having enough money for food), the coefficient reduces to 0.13. When regressions for different population sub-groups are examined, the coefficient for log income is very similar for men and women (around 0.15), but much larger for those of working age (around 0.18) in comparison to the youth and the elderly (around 0.10). Drawing on a much larger number of countries involved in the Gallup World Poll (125 in total), Helliwell, Barrington-Leigh, Harris and Huang (2010) found coefficients of around 0.4 for log household income. Their analyses controlled for a wide range of variables, including demographics, social connections, religion, perceived corruption, charitable giving (time and money) and food inadequacy (not enough money for food), as well as GDP per capita and a national-level measure of food inadequacy. The food inadequacy measure was defined net of its strong and significant correlation with household income - and this raised the estimated coefficient on household income.
Box 4.7. The range of income estimates observed in the life satisfaction literature (cont.)
- • Frijters, Haisken-DeNew and Shields (2004) looked at the effect of the large increase in real household income in East Germany on life satisfaction following the fall of the Berlin Wall in 1989. In the 10-year period between 1991 and 2001, the authors estimated that around 35-40% of the observed increase in average life satisfaction was attributable to the large (over 40%) increase in real household incomes during this time period, with a one-unit increase in log income corresponding to around a 0.5 unit increase in life satisfaction for both men and women.
- • Dolan and Metcalfe (2008) examined the life satisfaction impact of an urban regeneration project in a quasi-experimental design, which involved comparing two different communities in Wales. Background variables such as gender, age, relationship status and employment status were controlled in analyses. Whole sample analyses failed to find a significant effect of log household income on life satisfaction. With analyses restricted to individuals of working age, coefficients for household income were observed in the range of 0.65 to 0.93. These authors also note that in additional analyses, controlling for measures of social capital reduced the income coefficient by a non-trivial amount.
- • Although hard data is rarely reported, it appears that a relatively high proportion of individuals refuse to answer questions about their income in non-official and telephone-based surveys. For example, Smith (2013) estimated that income non-response rates ranged from under 5% to just over 35% for countries involved in the 2008 European Values Survey, with a similar variation in the World Values Survey. Gasparini and Gluzmann (2009) found non-response rates in the 2006 Gallup World Poll among Latin American and Caribbean countries to range from 2% (in Ecuador) to 39% (in Trinidad and Tobago). The potentially non-random nature of income non-responses (Riphahn and Serfling, 2005; Gasparini and Gluzmann, 2009; Zweimuller, 1992) and the various techniques deployed to manage them (which range from dropping all observations from the analysis, to imputation methods) can potentially impact on estimated coefficients. Further research and reporting on this phenomenon is needed.