RESEARCH DESIGN: THE INDICATOR-BASED EVALUATION FRAMEWORK
The research was framed as an exploratory case study of public RD&D financing of LCET in the Nordic countries. The study is constructed around indicator- based evaluation, confronting a literature review of indicators in RD&D policy evaluation with the actual usage of indicators in the Nordic countries. In order to enhance the understanding of indicator choice, an assessment of the indicator-based evaluation method was performed. Both primary and secondary data were collected to understand the respective funding instruments, their performance, and indicators used for their evaluation.[1]
Conceptualization of Indicator-Based RD&D Policy Evaluation
The multitude of indicators that is used in the assessment of RD&D policy can be categorized in different ways. A common differentiation is made between input, outcome, and impact indicators (Fischer 1995; Guedes etal. 2001; Neij and Astrand 2006; Miedzinski etal. 2013). Another (complementary) approach to conceptualize the use of indicators is to view them as a way to operationalize criteria for policy evaluation (Mickwitz 2003). Relevant criteria that were used to structure this study are administrative capacity, effectiveness, and additionality.
It is debatable whether administrative capacity should be seen as an evaluation criterion as such or as a ‘determinant of implementation’ (Vedung 2000: 226). Following the IPCC (Intergovernmental Panel on Climate Change) (Kolstad et al. 2014), it was used as a criterion in this study. Effectiveness refers to the degree to which ‘achieved outcomes correspond to the intended goals of the policy instrument’ (Mickwitz 2003: 426). For this study also indicators were included, which were de facto used to assess the effects of a financing instrument but for which no explicit goals were formulated. Due to the large number of potential indicators for effectiveness this criterion is frequently subdivided into environmental effectiveness, technological progress, and commercial effectiveness (Carlsson et al. 2002: 243; Carley, Brown, and Lawrence 2012: figure 2). The additionality criterion complements effectiveness. It is the degree to which achieved outcomes differ from a baseline development that assumes the absence of the respective policy instrument. The challenge of attributing specific developments to individual policy instruments is large (Scriven 1991). Still, additionality is a core criterion to establish accountability for the success or failure of RD&D support policies.
- [1] Further information about the case study, data collection and limitations of this study can befound in Sonnenschein (2016).
