Policy evaluation and cost-benefit analysis

The third main use of subjective well-being measures is to assist in the evaluation of policies. This includes both the direct use of measures of subjective well-being in formal policy evaluations as well as the more indirect - but possibly more important - role that they can play in cost-benefit analysis.

Box 1.2. Using measures of subjective well-being to value life events

People intuitively compare different life events on a daily basis and make judgements about how bad or good things might be. However, trying to put a number on the relative magnitude of the impact of different life events such as marriage or divorce, on a person's well-being - much less a monetary value - might seem counter-intuitive to many people. Nonetheless, such values are of potentially high interest from the perspective of thinking about how much to invest in preventing or encouraging a particular outcome.

Measures of subjective well-being provide a relatively straight-forward way of comparing the relative impact of fundamentally different life events in a quantitative way and, based on this, assigning such events a monetary value. Clark and Oswald (2002) present a method for valuing life events and, although the literature on using measures of subjective well-being to value life events has expanded significantly since 2002, the basic methodology remains largely unchanged. Consider the results below from a regression of a range of possible determinants of subjective well-being against life satisfaction (Boarini et al., 2012). The coefficients for the (base two) logarithm of household income, being married, and being unemployed are shown, and express the change in life satisfaction (on a scale of 0 to 10) associated with a doubling of income, being married, or being unemployed, respectively, holding all else constant.

Using these coefficients, it is possible to calculate the relative impact of being married compared to being unemployed on life satisfaction as 0.2584/0.4643 = 0.5565. Or, put more simply, being unemployed has almost twice the impact on life satisfaction as does being married.

Going beyond this, the monetary value of being married or being unemployed can be calculated by comparing the relevant coefficients with that associated with the coefficient for household income. Using the values presented above, the coefficient on being married is 0.2584/0.1482 = 1.7435 times larger than the impact of a doubling of household income. For a person with a household income equal to the OECD per capita household disposable income (USD 17 286 at PPP, 2008), this is equivalent to 1.7435 x USD 17 286 = USD 30 138. For unemployment the comparable value is 2.930 x USD 17 286 = USD 50 647.

These values are intended to illustrate the techniques involved, and need to be treated with caution. In particular, better measures would use panel data to capture the causal relationship (as do Clark and Oswald) rather than just correlation, and need to consider any potential biases in the data as well as the structure of the regression equations used to calculate the coefficients (Fujiwara and Campbell, 2011).

In formal policy evaluations, measures of subjective well-being can complement other social and economic indicators as a measure of the outcomes achieved by a policy. Here, as is the case with monitoring the progress of entire communities, measures of subjective well-being can add additional information over and above that captured by more traditional indicators. For some initiatives - where the impact on the subjective experiences of the population is the main object of the programme - measures of subjective well-being may even be suitable as the primary metric for assessing the programme’s success.

Many policy evaluations already include subjective measures of client satisfaction and questions on the respondent’s perceptions of what elements of the programme were most valuable. More general measures of overall subjective well-being, however, have some significant advantages over and above these more focused measures. Most importantly, measures of subjective well-being provide information on the actual impact of an initiative on the respondent’s subjective well-being, rather than the impact that the respondent consciously identifies. These values can differ because people’s judgements about the impact of a programme may be influenced by the fact that they have participated in the programme (i.e. they might be more prone to assign the cause of any recent changes in their well-being to the programme rather than to other factors, knowing that this is what he/she is being asked about). Also, people may not be aware of all of the various feedback loops via which a policy programme affects them. For example, in evaluating an active employment programme, respondents might consider the direct effect on their well-being of both having a job and gaining additional income, but not the flow on well-being that would stem from changes in their time-use due to longer commuting. Because measures of subjective well-being can capture the overall impact of a change on life circumstances, without requiring a cognitive judgement by the respondent on which causal pathways are being asked about, such measures provide useful additional information on the overall impact of a programme.

In some cases, measures of subjective well-being can be better than conventional cost-benefit analysis at treating non-monetary outcomes. Examining the relative costs and benefits of a proposal is relatively straight-forward when the proposal is aimed at strictly economic outcomes, and the costs and benefits of the proposal can be obtained from the relevant market prices. However, where the aim of a proposal is to achieve outcomes that do not have an obvious market price, it is much more challenging to obtain meaningful values for analysing the relevant costs and benefits. Because much government policy is concerned with market failures, many government policies are correspondingly focused on achieving non-market outcomes.

The traditional economic approaches to cost-benefit analysis for non-market outcomes depend on either revealed preference or contingent valuation techniques to estimate “prices” for such outcomes. A revealed preference approach involves calculating values based on the shadow prices implied by observed behaviour, while contingent valuation techniques calculate values based on the “willingness to pay” for the outcome in question, as expressed by respondents to a hypothetical question in a survey. Clarke and Oswald (2002) note that measures of subjective well-being can provide the framework for such valuations by comparing the impact of a particular outcome on subjective well-being with the impact of a change in income on subjective well-being. By making such a comparison, it is possible to calculate the amount of money required to achieve the same increase or decrease in well-being as that caused by the outcome under assessment.

There is good reason to believe that, in several circumstances, measures of subjective well-being have advantages over both revealed preference and contingent valuation for the purposes of cost-benefit analysis (see Box 1.3). An obvious advantage is that many measures of subjective well-being - such as overall life satisfaction - are relatively easy and

Box 1.3. The Green Book and life satisfaction

The Green Book is the formal guidance from the Treasury of the United Kingdom to other UK government agencies on how to appraise and evaluate policy proposals. The current edition of The Green Book dates to 2003, and provides advice on how officials should provide justification for a proposed government intervention, set objectives for the proposal, appraise the various options, and evaluate the effectiveness of the final action that results. In July 2011, The Green Book was updated to reflect the results of a review of valuation techniques for social cost-benefit analysis jointly commissioned by the Treasury and the Department for Work and Pensions (Fujiwara and Campbell, 2011). The review specifically focuses on the contribution that can be played by measures of subjective well-being - particularly life satisfaction - alongside more traditional approaches to cost-benefit analysis. In summarising the conclusions of the review, The Green Book states (p. 58):

A newer, "subjective well-being approach" has been gaining currency in recent years. The "life satisfaction approach" looks at people’s reported life satisfaction in surveys such as the ONS’s Integrated Household Survey, which began including questions on respondents’ subjective well-being in April 2011. The life satisfaction approach uses econometrics to estimate the life satisfaction provided by certain non-market goods, and coverts this into a monetary figure by combining it with an estimate of the effect of income on life satisfaction.

At the moment, subjective well-being measurement remains an evolving methodology and existing valuations are not sufficiently accepted as robust enough for direct use in Social Cost-benefit Analysis. The technique is under development, however, and may soon be developed to the point where it can provide a reliable and accepted complement to the market based approaches outlined above. In the meantime, the technique will be important in ensuring that the full range of impacts of proposed policies are considered, and may provide added information about the relative value of non-market goods compared with each other, if not yet with market goods.

While the amendment to The Green Book stops short of fully endorsing the use of life satisfaction measures for use in formally evaluating government programmes, the decision to make an interim amendment in itself signals strongly the importance that UK central agencies attach to obtaining improved measures of the value of non-market outcomes.

cheap to collect. However, there are also more substantive methodological advantages that may be associated with using measures of subjective well-being in this way. Revealed preference relies on strong assumptions about people’s ability to know how an outcome will affect them in the future, and on the assumptions that markets are in equilibrium. Diener, Lucas, Schimmack and Helliwell (2009) note that for market prices for houses to reflect the disutility of airport noise accurately would require that house purchasers are able to forecast how much the noise will impact them before buying the house. Similarly, in this example, it is difficult to disentangle the differences in house prices due to noise from differences in other aspects of house quality.

Contingent valuation also relies strongly on people’s ability to make accurate judgements about how something will make them feel in the future. Dolan and Peasgood (2006) note that people have difficulty imagining how good or bad different circumstances are actually going to be. Indeed, the “willingness to pay” surveys commonly used for contingent valuation are, to a large degree, measures of the subjective well-being associated with a hypothetical scenario. Using measures of subjective well-being to calculate the costs based on the actual impact of different life circumstances on subjective well-being removes the hypothetical element from the equation. In addition, contingent valuation surveys tend to produce very different estimates of the value of outcomes for people at different points on the income distribution. This tends to result in either weighing the desires of the rich more heavily than the poor when assessing the costs and benefits associated with the proposal under consideration or taking account of the marginal utility of income in calculating the final cost. The latter approach is difficult in the absence of robust estimates of the marginal utility of income (Dolan and White, 2007).