Mixed-methods approach

Process tracing

Process tracing was applied in order to trace the evolution of military transformation and changes in civil-military relations between 2002 and 2017. This is done by triangulating observations from multiple sources, i.e. interviews, survey responses, web content, to assess the dynamics and conditions of change from q (2002-2008) to t2 (2008-2017). The formal interview and survey data are complemented by numerous examples capturing the developments post-2017. By providing empirical evidence, e.g. respondent statements, to describe relevant sequences, moments or regularities, the analysis in this book allows us to trace the processes of change and estimate, through a systematic analysis of descriptive and more causal inferences (Collier 2011), relevant variables and intervening factors (conditions or determinants). This analysis does not claim determinacy or non-exhaustivity. It rather reflects the perceptions of the respondents, as they were in 2017, when the field research was conducted.

A major utility of using process tracing is its capability to capture the complexity of ‘causal sequences’ and trace ‘micro-causal relationships’ (Ruback 2010: 478-9) and a variety of intervening factors. Causal-process observations (CPOs) (Brady and Collier 2010; Collier el al. 2010) can be inductively derived from the interview and survey data and employed for generating new theoretical inputs. Process tracing along the longitudinal case facilitates the identification of interactive effects, which are not “independent of each other”, as well as path-dependent processes (George and Bennett 2005: 208). The rich variation of events and historical developments in Pakistan is expected to facilitate the identification of chains of influence and explanations for civil-military outcomes. In order to increase the validity of the results, attention was also accorded to alternative explanations (Bennett and Elman 2007: 183) and numerous examples from other cases. Counter- factual analytical inferences were made in order to assess the relevance of the intervening factors for the outcome. Counterfactual analysis can have a particular contribution for additional clarifications as well as for the theory development endeavour, by identifying “critical variables” (George and Bennett 2005: 232).

Through its epistemic innovation, process tracing allows unfolding both the ‘logic of appraisal’ and the ‘logic of discovery’ (Popper 2005). On one side, inferences of interest can be tested by applying a focused approach. On the other side, theory-building inputs can be generated by employing an inductive mode of reasoning and multiple data sources (data triangulation). Contrasting a monistic interpretation, Aristotle argued that outcomes can be determined by several types of causes: formal, material, efficient and final (Stein 2011). Process tracing techniques enable the identification of possible “equifinality” or multiple causality, i.e. alternative paths leading to similar outcomes. Considering the geographical outreach of the collected data, the rich variety of factors might make it possible to address “insufficient but necessary parts of a condition which is itself unnecessary but sufficient” (INUS conditions) for the outcome (Mackie 1965: 245). This pluralistic perspective facilitates a differentiated typology for theory development (George and Bennett 2005) in the ‘building blocks’ sense. Even in single-case designs, process tracing admits ‘contingent generalisations’ (George and Bennett 2005: 217), from which middle-level theories can be derived.

Content analysis

Content analysis is used for the analysis of the in-depth interviews, whose content was transposed into categories based on a pre-defined codebook using the software N Vivo. The interview transcripts comprised over 100,000 words. Content analysis is a suitable methodological technique for processing information and extracting meaningful, significant and representative knowledge from large volumes of data (Krippendorff 2013: 46-7).

With its epistemological emphasis on Erklaren, content analysis allows us to explain why some organisations are more successful in their endeavour than others in interacting with the military and to compare the ability of different civil society actors to perform their work. In other words, what are the major determinants of civil-military interactions, e.g. is the type of CSO or their vision/strategy of engagement a determinant of their ability to engage in democratic synergies with the military?

A further utility of the content analysis is the increased reliability, validity and generalisability of the results for further cases. Formal measures of reliability such as intra-coder reliability or ‘reproducibility’ (Stemler 2001: 5) were applied in order to assure increased credence to the results of the empirical analysis. A formal inter-coder reliability test was also applied. During the inter-coder reliability test, the author’s co-supervisor coded a significant sample (45 references) of data applying the same codebook. The average rate of agreement was 88%, with an average Kappa coefficient of 0.71 (see Annex 2). These results suggest that the data were coded in a reliable manner.

 
Source
< Prev   CONTENTS   Source   Next >