Behind the Scenes of the French Evidence-Based Policy Movement: Rise and Fall of an RCT-Driven Model of Evidence
The beginning of the new century witnessed a growing worldwide interest in impact assessment by evaluation methods, and a general call for Evidence-Based Policy (EBP) within international and national organizations (Donaldson, Christie, & Mark, 2009; Evaluation Gap Working Group, 2006). EBP movements call for policy initiatives to be more systematically supported by evidence, and they create opportunities for dialogue between those involved with the supply and demand for that evidence (Sanderson, 2002). The strategies of these movements are two-fold. First, they hope to act upon the supply side of evidence by improving the quality of studies produced; second, they attempt to foster the use of evidence by policy actors by institutionalizing a more systematic use of evidence in policymaking (Lee, 2004; Parsons, 2002). An overview of EBP movements around the world shows that their strategies vary depending on the characteristics of national evaluation suppliers, national institutions, and political culture (Rieper, 2009). Across countries, a diversity of methods for impact evaluation has been promoted, different types of suppliers of evaluation have been identified, and various types of knowledge synthesis are being delivered to policymakers.
Research literature in this field has extensively covered scientific debate on the most relevant methods of producing evidence, as well as on how to identify barriers and facilitate use of evidence by policymakers (Donaldson et al., 2009; Duflo, Glemierster, & Kremer, 2004; Sanderson, 2002; Solesbury, 2001). However, little attention has been given to the study of EBP movements themselves and to accounting for national choices of EBP in regard to standards of evidence, both on the supply and the demand side. This study expands upon this body of research by looking at the roots of EBP movements, their genesis, and emerging EBP coalitions.
This research1 adopted a Political Sociology of Science2 approach to account for variations in EBP models across countries (Frickel & Moore, 2006). This approach calls for devoting special attention to the individuals that participate in these movements, and the interests driving them to champion particular EBP models. It leads us to understand EBP movements as indeterminate processes under the influence of many actors, both from the supply and the demand side, pursuing various aims, with conflicting interests at times. Consequently, the hypotheses put to the test are: 1) that national EBP models emerge as consensuses between these actors which reflect their positions, and 2) that one can therefore also expect these consensuses to be subject to instability when there are shifts in the balances of power by the actors driving supply of and demand for evidence.
To test these hypotheses, the present chapter undertook a thorough study of the French EBP movement from 2005 to 2013. This movement is of particular interest in many regards. First, it was subject to rapid institutionalization, as only four years separate the first mention of EBP by a public official in 2005, and the creation of a dedicated national fund within the French government for this purpose in 2009. Second, because the French EBP model explicitly promoted the use of Randomized Controlled Trials (RCTs), despite the infrequent use of this method by French evaluators and scientists. Third, because the French EBP movement, although promising, has in hindsight led to very little use of evidence from impact evaluation, and has rapidly vanished from use by the central government, for reasons which we hope to clarify.
Data used in this research were collected in semi-structured interviews conducted with more than 50 high-level French administration officials, scientists who had conducted RCTs, and managers of pilot programs. These interviews were complemented by a literature review of approximately 200 scientific and administrative documents that were produced by members of the French EBP coalition. Finally, 15 in-depth case studies were conducted of RCTs implemented in France.
Evolution of EBP Models and the Role of RCTs in France
This first section presents why and how a coalition for EBP emerged in France, and how the actors involved influenced the shape of EBP. Studying the interests and strategies of individuals within this coalition helps explain not only the methodological choices of the French EBP movement, but also the limits this placed on its sustainability.