"Trust in numbers" and the appropriateness of broadbrush risk analysis

The perceived legitimacy-enhancing capacities of risk analysis are founded on wider set of claims about “trust in numbers” (Porter, 1995). These suggest that the design of risk analysis does not even have to be rigorous to fulfil goals of institutional risk management. I will detail this logic briefly, as it matters greatly for the empirical identification of legitimacy-seeking in analytical tool change. Porter describes quantification in administrative decision-making - risk-based or otherwise - as a strategy of overcoming a lack of ascribed expertise to do the job. Where authority cannot, or can no longer be claimed based on professional training, expertise, and judgment alone, quantification becomes an important “technology of trust” (p. 15):

A decision made by the numbers (or by explicit rules of some other sort) has at least the appearance of being fair and impersonal. Scientific objectivity . . . provides an answer to a moral demand for impartiality and fairness. Quantification is a way of making decisions without seeming to decide. Objectivity lends authority to officials who have very little of their own.

(Porter, / 995, p. 8)

Numbers appeal, according to Porter, because “[t]hey provide legitimacy for administrative actions, in large parts because they provide standards against which people judge themselves” and their performance (p. 45). Drawing on the work of Jasanoff, Porter highlights that a culture of adversarial litigation in the US has created a climate of mistrust toward the administration in which numerical justification could blossom particularly well (Porter, 1995, pp. 194-199). Legitimation by numbers appeals in contexts where there is a widespread mistrust of an administrative body or regulator and where the accountability pressure on their mandate is particularly high. Boswell (2018) develops a similar argument about how measurement - by signaling rigor, commitment and the willingness to be held accountable against numbers - can generate political trust in a public administration (but she also explores the problems of discursive lock-ins to constant performance measurement for the British case).

Porter’s vivid example is the introduction of CBA in the early 19th century by the US Army Corps of Engineers - a body responsible for planning and executing drainage, levee, and canal projects (also see Chapter 4). Unlike usually suggested by instrumental accounts today, the initial adoption of CBA was not at all related to hopes of efficiency gains but was “an attempt to create a basis for mutual accommodation in a context of suspicion and disagreement” (p. 148). In an environment “of political pressure and administrative conflict . . . cost-benefit methods were introduced to promote procedural regularity and to give public evidence of fairness in the selection of water projects” (p. 149). In this reading, the Corps turned to quantification because they could not rely on a great deal of externally ascribed authority in an administrative climate of “utter disunity and savage infighting” (ibid.). CBA became a strategy of convincing Congress, the Courts, and potential opponents in competing departments that the Corps’ funding decisions on water projects were reasonable, fair, and politically unbiased. It became a tool for legitimating the Corps as a public actor.

While Porter uses the example of CBA, analogous claims have emerged for risk analysis, too. Actuarial assessment offers a way of quantifying phenomena and generating seemingly objective options for otherwise contested decisionmaking. Indeed, the well-charted methodological and practical limits to rigorous problem-oriented risk analysis (Black, 2010) do not impede its widespread application in “justifying] governmental regulation” (ibid.: p. 314). Hutter (2005) argues that risk analysis can legitimize administrative action not because of its rigor but because it is commonly accepted as an appropriate, business-like way of justifying choices: “their apparent objectivity and transparency could be used to explain the allocation of resources, in a way which was well tested and trusted by the business community” (p. 2f). These readings resonate with the arguments of institutional isomorphism, especially mimicry, rehearsed in Chapter 2 (DiMaggio and Powell, 1983).

Central to Porter, Black, and Hutter’s analyses is the observation that the actual scientific rigor of CBA or risk analysis does not matter much for the fulfilment of their legitimizing promises. Indeed, Porter (1995, p. 158) describes a strong “implicit faith” among political decision makers in the Corps’ CBA reports and the political independence of the Corps. Anecdotes about parliamentary hearings - where Congress may well have inquired the numbers and “asked many factual questions” about CBA, “but it rarely mattered what the answer was” (Porter, 1995, p. 157) - indicate that the authority of the Corps’ decisions hinged on a generic trust in numerical assessment. Numbers were perceived as a more objective foundation for decisions than mere professional judgment (a similar argument is developed for the case of performance targets in the UK public administration by Boswell, 2018). However, parliamentary or judicial reviews did not usually provoke any detailed engagement with the assumptions and methodological decisions that underpinned any one CBA.

In summary, “trust in numbers” amounts to a rather unquestioned faith in the legitimacy of numerically founded decisions. The design of a risk analysis template and its ability to solve risk problems will thus be secondary to its rhetorical/sym-bolic ability to convince critiques of a fair and rational decision-making process. In terms of policy design, where administrators are mainly worried about the legitimacy of their mandates rather than the alleged problem-solving functions of risk analysis, there will be less reason to invest in the depth, uniformity, and scientific rigor of their risk-analysis template.

Indicators for actors' legitimacy-seeking orientations toward risk analysis

Overall, legitimacy-seeking orientations toward risk analysis pursue institutional risk management goals. Risk-based differentiation will be used to define the limits of one’s own accountability and “blameability” vis-à-vis others. As cost-effectiveness is not the chief goal here, the need to design a detailed, uniform, and scientifically rigorous model for risk analysis is lower than in problem-solving contexts.

Legitimacy-seeking turns to risk analysis will feature the following goals:

  • • institutional risk management
  • • limiting potential to be blamed for adverse outcomes or shifting blame to others

In empirical applications, legitimacy-oriented risk analysis fulfils the following roles:

  • • differentiation of risk scenarios to signal fair, objective, and effective problemsolving to others
  • • risk-related signaling of one’s limits of accountability for adverse outcomes

The technical/organizational design of legitimacy-oriented risk analysis will involve the following:

  • • no strong focus on scientific rigor and independence
  • • broad and/or alternative risk analysis templates with room for discretion (including for the assessment of likelihood and impact of adverse events)
  • • relatively broad guidance on design and use of risk analysis, leaving room for discretion.
< Prev   CONTENTS   Source   Next >