Evaluating Labor Market Policies

The evaluation of policies and political measures has been discussed in general in subsection 6.8 above. The example of the evaluation of labor market policy and the according measures shall be discussed a little more detailed, as it is one of the basic utilizations for labor market indicators.

Labor Market Policy Evaluation

The approach to labor market policy and its evaluation is similar to the approaches discussed in the chapter above. A certain target is set; an appropriate policy is chosen which is then implemented. The effected measures then lead to the take-up of the policy, i.e., specific actions are carried out within the program or measure. This affects the labor market and therefore the target which can, in turn, be evaluated (cf. Figure 51). This proceeding is simplified, though. It neglects, for example, exogenous influences that may have an impact on the target.

Figure 51: The policy evaluation cycle

(Schmid 1997)

The simplified approach to labor market policy evaluation is the goal- or target- oriented evaluation described in subsection 6.8. This method adds to the traditional, program oriented evaluation the possibility to include processes of the political development of the measure as well as the socioeconomic context and the conditions of the implementation, allowing the inclusion of secondary objects into to the evaluation process (Bangel et al. 2006).

But even in the first step, the setting of the target, evaluation is relevant. It must not only assess ex post whether targets have been reached, it must also discuss ex ante, whether the targets are appropriate and clearly defined. This includes the discussion whether there is a plurality of measures that may influence the same issue in opposed ways (BLASCHKE, Plath 2000). To better understand the process of the implementation and evaluation of labor market policy measures, it is helpful to gain an overview over the organization, i.e., the institutional basics, of labor market policy. Generally, there are three elements of institutional frameworks for individual policies (cf. Figure 52).

• (1) Policy regimes, i.e., the policy outcome of the different campaigns of the groups involved;
• (2) Organizational regimes, i.e., the element providing the financial framework and the administrative infrastructure necessary for the implementation of programs and measures; and
• (3) Incentive regimes, i.e., context conditions influencing the “adoption of policies in the target area” (Schmid 1997).

Figure 52: Analytic framework for the evaluation of labor market policy organization

(Schmid 1997)

For each of the elements there are a number of organizational structures. Policy regimes, for example, can be individualist or communitarian, libertarian or social etc; whereas implementation regimes can be, e.g., demand- or supply-oriented, private or public, or target- or rule oriented. Incentive regimes aim for example at either intrinsic or extrinsic motivation, high or low financial incentives or concentrate on status hierarchies or functional hierarchies. Depending on the regime, there are different performance indicators. For policy regimes, the responsiveness is monitored, implementation regimes are assessed by their organizational efficiency; and incentive regimes can be evaluated for their effectiveness and efficiency. The questions regarding LMP and the according policy measures which evaluation is expected to answer concern, for example, the effectiveness and efficiency of the programs and measures. To determine those dimensions, the impact and the costs (including opportunity costs as well as the costs for possible external effects) must be measured. The measurment of impacts proves a formidable challenge, though. Traditional LMP evaluation focused on individual policy instruments and programs; mostly by comparing the goals of the programs with the outcomes. In those traditional evaluations, the formation of the policies and the process of implementation are treated, as Schmid (1997) calls it, as a black box. The comparison of goals with outcomes is often conducted as a basic cost-benefit analysis. If costs are higher than the benefits, then the project is considered as not successful. If the benefits are higher than the costs, the project was successful, but can still be less successful than a different measure with an even better cost-benefit balance. In theory, it would also be necessary to measure whether the non-execution of a program or measure would have led to even higher costs through inaction. If this were the case, then a measure that has higher costs than benefits, but a better balance than that of the non-program case, may still be continued for a certain time. But cost-benefit analysis has a distinct disadvantage: it assumes that costs and benefits are measurable in a common unit (preferably monetary). As has been discussed in subsection 4.1, benefits of labor market policy do not only regard direct influences on individual and national income, but also on individual wellbeing (cf. subsection 5.7), an aspect that is only measurable in some of its direct results (e.g., better health, thus lessening the costs for the health system), but not in its entirety and not in monetary units.

In addition, there are other risks of traditional evaluation concepts. First of all, success indicators tend to concentrate on the supply side of the measure, i.e., on the continuance of participants and the incorporation rate into the first labor market, thereby ignoring possible changes on the demand side, namely the regional labor market situation or socioeconomic determinants of the labor market development. Secondly, to compare the impact of different policy measures, evaluators tend to concentrate on a number of basic quantifiable indicators, thereby neglecting special characteristics of the particular program. Thirdly, there is a danger of confusing financial control, evaluation and monitoring. Fourthly, the risk of counterproductive impact of evaluation has to be kept in mind. This is, in general language, often called “learning for the test”, i.e., figuratively the tailoring of projects and measures to meet the targets for evaluated indicators (e.g., by constructing measures only for the unemployed most capable of competing on the job market), thereby risking the neglect of other dimensions or aspects that have been overlooked in the choice of indicators. Fifthly, there is a risk of neglecting complex interwoven institutional situations (e.g., competition with other labor market programs, difficult implementation, etc.) as well as risk of neglecting superior purposes while concentrating on minor, attributable targets (Bangel ET AL. 2006).

In summary, it is necessary to acknowledge that the evaluation of active labor market policies is not comprehensive; and that it is impossible to draw absolute conclusions from the data researched. What evaluation can do, for example, is to show that policies can have a positive impact; labor market policy is therefore not useless. Also, it can show that a well-chosen combination of measures leads to the best results (Im- MERVOLL, SCARPETTA 2012). As for the actual execution of labor market policy in Germany, it can be said that in general, it does not differ much from the development of evaluation as a whole. Starting from the 1980s, the German Institute for Employment Research (IAB), for example, conducted studies to evaluate immediate and continuing effects based on data accrued in the process of administration (Blaschke, Plath 2000). In 2003, the German Ministry for Economy and Labor (BMWA) authorized a project group for evaluations in the Ministry. The group was expected to assess previous evaluations realized in the ministry and to compare them with international evaluation standards; with the aim to know how a good evaluation should be performed and which expectations the ordering party may have (LORENZEN 2008). This project was a considerable step towards an institutionalization of evaluations, as in the beginning of the 1990s evaluation was not utilized in the Ministry at all. For laws and measures to be passed there were impact assessments; but those were not universally structured after objective, general criteria. This changed, though, and by now all laws and labor market programs have to be evaluated independently (Weiland 2008).