Data and methodology
Data used for the analysis is provided by the 'Comparative Public Organization Data Base for Research and Analysis', or COBRA-network. The COBRA survey was originally developed by the Public Management Institute at the Catholic University of Leuven by Geert Bouckaert, Guy Peters, Koen Verhoest and Bram Verschuere. The questionnaire focuses on issues of autonomy, control and management of public sector organizations, in particular (semi-) autonomous agencies, and targets the senior manager of the surveyed public sector organization as respondent. The dataset used comprises agency-level survey data from 15 different countries. The COBRA dataset includes all the variables needed for the analysis in the case of only three countries (Portugal, Belgium (Flanders) and the Netherlands), as some of the variables relate to survey questions that were not obligatory in the country-specific surveys. Including these three countries gives us a large enough sample to do relevant analyses and gives us a good representation of all types of agencies with enough variation in formal independence, managerial autonomy and size. The three countries are to a certain extent comparable, as their administrative law systems, which are still very important in shaping public administration and policy, are grounded in Napoleonic occupation. Moreover, the three countries are rather moderate reformers in terms of liberalization of markets and the introduction of NPM (Pollitt and Bouckaert 2011), although it might be argued that the Netherlands is more active in reforming than are the other two countries. The dataset only takes into account agencies that have regulation, scrutiny, control or inspection as their primary task and for which we have all data for the variables concerned, which amounts to 30 organizations in total. From 14 organizations with regulation as their primary task, we retained 8 organizations in the dataset in Belgium (Flanders); from 22 such organizations in the Netherlands, 10 were retained; and out of 20 Portuguese organizations with a regulatory task, we retained 12 organizations. The substantial drop-out is due to the use of a large number of variables in our analysis; all cases lacking a value for one of these variables were eliminated from the dataset. However, the retained sample of organizations allows us to confront our hypotheses with empirical material, as it has a good variation in terms of the conditions we want to study, reflecting the variation in the original dataset encompassing the three countries. Nevertheless, full representativeness cannot be assured, which makes it difficult to generalize the results from this analysis. Moreover, although the introductory chapter pointed at the explanatory value of country and policy sector in order to understand the accountability of regulatory agencies, we cannot include the country as a condition as FsQCA requires some kind of ordinal ranking of countries. However, we did label the country of the cases showing relevant paths in order to check ex post for prevalence of (cases out of) the three countries in paths, and as such for the explanatory power of 'country'. Moreover, our sample contains agencies from a very wide variety of policy sectors, making it impossible to use sector as condition in our analyses and to study the explanatory value of policy sector. In that regard the analysis in this chapter mainly has explorative value, producing findings that need to be tested for their generalizability in further research.
In this chapter we apply fuzzy set qualitative comparative analysis (FsQCA2), which is a set-theoretic approach (Ragin 2000). Set-theoretic approaches describe causal complexity in terms of relationships between conditions (in frequentist methods: independent variables) and an outcome (in frequentist methods: dependent variable). The assessment of causal complexity in set-theoretic methods is based on a few assumptions:
- (I) Conjunctural causation; a condition will only have an effect in combination with other factors.
- (II) Equifinality; an outcome can be elucidated by multiple, mutually non-exclusive (paths of) conditions.
- (III) Causal asymmetry; the presence of the outcome may have different explanations than its absence.
These enlisted assumptions are clearly different from statistical methods (Schneider and Wagemann 2012). In this chapter we are mainly interested in how different factors together, in interaction (that is, configurations of conditions), bring about a certain outcome. Less important, but notable, is that given our limited sample size (n=30), standard frequentist methods for comparison can be considered to be too unsophisticated.