The Practical Assessment: Governance Indicators and Indices—What is Measured and How?

Departing from classical state-centred perspectives as described above, the multidimensional governance concept became 'a concept of many proprietors and many varieties of definition and explanation' (Rotberg 2014, p. 511). Thus, as outlined above, it represents a patchwork of collective decisionmaking structures and processes as well as purposeful collective action for public and societal issues that render a uniform definition and measurement practice quite problematic (Peters 2012, p. 29). So, while 'definitional challenges should not lead to paralysis in measurement' (Kaufmann and Kraay 2007, p. 2), differences in measuring governance, 'good' governance, and global governance are to be expected. This is especially true for the development of sound measurement concepts as 'sound theoretical concepts are crucial if such a complex phenomenon as democracy [or governance] is to be made measurable' (Giebler 2012, p. 510).

In general, measuring governance aims at quantifying and qualifying the characteristics of a key paradigm of the political, institutional, legal, cultural, and social reality, and practice within multilevel political interactions that are not 'countable' per se. Due to its conceptual 'elasticity' and lack of 'specificity' (Doornbos 2001, p. 95), existing measurement practice relies on aggregations of indicators and data that construct countable and measurable objects to approach the phenomenon. Therefore, concrete measures of governance reflect the particular political, institutional, legal, cultural, social, and geographical contexts in which they are developed and employed.

Due to the multiple environments and levels at which governance indicators appear and are used, their conceptual nature varies according to their particular purpose, usage, and developer/user groups. These variations in definition make governance measurement a popular, yet contested instrument of global, regional, national, and local reach. Although according to Giebler, 'one might even come to the conclusion that the "competition" between different approaches is exclusively fought on the level of theory or definitions... neglecting the importance of adequate concept operationalization' (2012, p. 510) would be very problematic. Especially composite indicators strongly mirror this definitional ambiguity as they,

frequently combine incongruous concepts. [So, w]hile they make for nice headlines and facilitate regression analysis, they provide little actual information that can be used by governments, practitioners, and researchers to understand the true drivers of capacity and what can be done to effectuate change.

(Holt and Manning 2014, p. 723)

The contestation around governance measures is, moreover, increased by the fact that they are expected to perform in very different settings for largely different actors (Bovaird and Loffler 2003, p. 316). The ends to which they are employed, that is, to evaluate and/or reform governance structures and processes as well as institutions, remain, however, largely similar across the different usages. This issue creates fundamental challenges not only for the framing and design of governance measures, but also for the adaptation and modernization of governance systems as a consequence of governance evaluations. Assuming, however, that governance measures could be defined homogenously, would repeat 'the error of many broad governance indicators producing measures detached from any clear concept of what they are measuring' (Holt and Manning 2014, p. 719).

Regardless of these uncertainties surrounding measurement methods, governance measures are highly influential. Even if different from authoritative regulatory instruments, they put pressure formally and informally on actors' behaviour and thus influence change. Donor organizations offering development support, in particular, prominently use these tools to steer their aid allocation policies as 'investors increasingly want back-up to traditional, macroeconomic-based, country-risk indicators that have failed to predict costly financial crisis' (Arndt 2008, p. 277). Examples such as the World Bank's Worldwide Governance Indicators and the Doing Business indicators rank high on this list. With this political and financial relevance, governance measures at the same time represent part of what they are set out to measure: diverse modes of governance within the multilevel institutional setting of globalized politics.

The development of governance measures got boosted when the attention generally shifted from the 'excellence in service delivery to good governance' (Bovaird & Loffler 2003, p. 314). This move was motivated by the increasing analysis of changes in classical nation state structures and processes and the acknowledgement 'of "wicked problems" which do not lend themselves to solution simply by service improvement in each of the agencies concerned' (ibid.). As a consequence, since the mid-1990s 'a veritable explosion of interest in the quality of "governance"' (Arndt 2008, p. 275) could be witnessed and governance indicators as well as indices mushroomed as a result. Many of these tools have been developed by international organizations such as the World Bank (Worldwide Governance Indicators; Doing Business indicators) or the UN (Human Development Index), but also non-governmental organizations or private actors, such as Transparency International (Corruption Perception Index), Freedom House (Freedom in the World), the Bertelsmann Foundation (Sustainable Governance Indicators; Transformation Index), and Global Integrity (Global Integrity Report).

When looking at concrete indices and indicators of governance, a strong focus on measuring 'good' governance has to be acknowledged, while measuring 'global' governance is rarely done or even attempted. The existing landscape of 'good' governance indicators is hence broad, sophisticated, and diverse. A brief analysis including some examples of the most well- known of these measures and their key features and differences shows a remarkable heterogeneity, but also some common features (see Table 3.5; Malito 2014).

Existing measurement tools differ in their particular focus, scope, aims, and measurement targets. The primary geographic level of reference is the national level, meaning that these tools measure governance at the national level or

Table 3.5. Main Examples of Governance Indices and Indicators

Source: Adapted and amended version of Malito 2014, p. 7.

below. Some measures relate to the quality of democracy and its institutions (such as Freedom in the World of Freedom House), while others measure the functioning and quality of public administrations (like the Bertelsmann Transformation Index and Sustainable Governance Indicators). Again others assess state capacity or the rule of law (for Worldwide Governance Indicators; see Fukuyama 2013, p. 348f.). Overlaps between these different measurement targets are not uncommon among the different tools and therefore some rule of law or state capacity measures also assess the quality of public administration. Hence, while some measures monodimensionally focus on only one feature of governance (such as the World Justice Project's Rule of Law; Transparency International's Corruption Perception Index), others combine several elements and are hence regarded as multidimensional tools (World Bank's Worldwide Governance Indicators or Country Policy and Institutional Assess- ments/CPIA; Bertelsmann Sustainable Governance Indicators). As Malito points out, 'mono-dimensional measures account for a single aspect or dimension of governance, while multi-dimensional indices integrate different representations of the same phenomena in the same measure' (Malito 2014, p. 6).

Within the development context, governance measures strongly embrace the normative aspects of 'good' governance and assess elements that are 'either instrumental (they lead to development outcomes we care about) or intrinsic (they are worth doing in and of themselves)' (Holt and Manning 2014, p. 726). The measurement of improved governance in the development context also especially targets at the control of corruption and public administration reform with 'democracy and good governance... [seen as] mutually supportive' (Fukuyama 2013, p. 350).

The further differentiation into input-/rules-based and output-/outcome- focused measures represents another key division of approaches to assess the quality of governance (Kaufmann and Kraay 2008, p. 4). Input- or rules-related measures, such as the World Bank's Doing Business indicators, focus on clear structural components of governance and assess the institutional arrangements in place to deliver public policies. They include administrative structures, institutions or legislation in place; human resources or IT equipment; and focus on the 'structural and management characteristics of governments' (Andrews 2008, p. 381; Bovaird and Loffler 2003; Fukuyama 2013; Holt and Manning 2014). This perspective particularly argues 'that measuring performance ... can best be done by using publicly available objective (not subjective) data, and by examining outputs (results), not inputs' (Rotberg 2014, p. 512).

In contrast to this, output- or outcome-related measurement tools focus on governments' achievements, meaning policy outcomes that governance structures and processes deliver. Key examples most often presented include public spending levels and distribution, literacy rates, education levels, life expectancy, mortality rates, GDP per capita rates, completed court cases, duration of administrative proceedings, and the like (all representing approximations to governance performance). This focus assumes 'that outputs and outcomes... represent social values and what citizens expect governments to deliver' (Holt and Manning 2014, p. 717). As such, they are viewed to constitute valuable and valid measurement units to assess governance performance. This perspective particularly supports the assessment of 'how governments are performing' (Fukuyama 2013, p. 355), and the analysis of 'integral benchmarks of performance' (Holt and Manning 2014, p. 717; Boardman 2014).

However, such an output and outcome focus is not unproblematic as the measurement of public services represents a huge methodological issue in itself, given that 'outcomes are often contingent on factors outside the direct control of the agency responsible for delivering particular outputs' (Bovaird and Loffler 2003, p. 317). In this vein, '[m]easures of rule of law, like time to trial, rate of case clearance, etc., say nothing about the quality of the justice being produced by a legal system' (Fukuyama 2013, p. 356). Therefore, focusing on output and outcomes can create some issues for concrete measurement according to Holt and Manning:

• (1) they can be affected by exogenous factors, making it difficult to isolate the contribution of public action; (2) measuring quality aspects is difficult; and (3) normative and procedural concerns (i.e., how the output or outcome was achieved) still matter, particularly in policy areas involving human and individual rights.
• (Holt and Manning 2014, p. 718)

The output- and outcome-related perspective is, moreover, problematic as it establishes causal link relations by taking outcomes as proxies for governance quality. In this way, outcomes are viewed as direct results of governance. They become attributed to public policies and institutions, hence to the input- or rules-based perspective. In doing so, the output- and outcome-related perspective yet neglects intervening factors that co-influence policy outcomes, such as the participation of societal actors in the implementation of public policies (Fukuyama 2013; Kaufmann and Kraay 2008). Therefore, this form of governance measure indeed excludes key structural-procedural governance features of interrelated multilevel policymaking and the integration of most different political and societal actors within it. Political arenas in particular that combine 'weak states and strong societies' where 'states could have equal regulatory capacity but unequal regulatory outcomes because the society in one is better organized to resist state penetration than the other' (Fukuyama 2013, p. 355 referring to Migdal 1988) are cases in point here.

Compared to this variety of governance and 'good' governance measures, the universe of global governance measurement is rather un(der)populated and—caused by differences in focus and understanding—this realm is still within the midst of its conceptualization phase. Interesting approaches are, for instance, brought forward by Biermann et al. that differentiate global governance structures by their level of fragmentation. Analysing their respective degree of 'institutional integration', 'norm conflicts' involved, and related 'actor constellations', different forms of 'global governance architectures' are differentiated ranging from 'conflictive fragmentation' over 'cooperative fragmentation' towards 'synergistic fragmentation' (Biermann et al. 2009, p. 19). In addition, different degrees of global governance performance are identified around the interrelated characteristics of speed, ambition, participation, and equity. The analysis of these degrees is 'organized around the question of (1) the relative speed of reaching agreements; (2) the level of regulatory ambition that can be realized; (3) the level of potential participation of actors and sectors; and (4) the equity concerns involved' (ibid., p. 24). Identifying such key features of global governance structures and processes, such conceptualizations could indeed serve as starting points for the development of global governance measures to assess existing arrangements at the global level.