A Comparative Study on Cost-Sharing in Higher Education—Using the Case Study Approach to Contribute to Evidence-Based Policy

Dominic Orr

Introduction

As Solesbury has stated, evidence-based policy is not so much the triumph of the social science—i.e. supply-driven—but expression of a greater pragmatism on the part of the policy development process—i.e. it is demand driven (Solesbury 2001). Evidence-based policy presents an opportunity, but also a direct challenge for researchers; that of providing evidence which is deemed relevant to and can be communicated and discussed within the policy-making process. This paper will present a project, which was commissioned by the European Union to provide advice to countries on changes to the balance of cost-sharing in higher education and their possible impacts (Orr et al. 2014a, b). This commission can be seen within the context of a multinational organisation encouraging the use of evidence-based policy for European and national decision-making in the policy sphere. In the case of the European Union, it set this as one of its strategic goals in 2002 (EU 2002). Indeed national governments appear to be increasingly persuaded by the argument that policies can be developed and evaluated based on the experiences of other countries, which have adopted similar policies. This is evidenced by frequent requests from national governments for such advice from multinational organisations, particularly the European Union and the Organisation for Economic Cooperation and Development (OECD).^[1] This means that enterprises, such as the study presented here, are expected to facilitate policy learning between countries; these studies endeavour to make a virtue out of the fact that similar policies are implemented in different ways in different contexts, providing a rich ground for learning about how they work. The very virtue of being able to observe “living social experiments” also presents a huge challenge for research. This is of making the object of investigation comparable—of creating “relationally equivalent phenomena” (Phillips 2006)—to enable a comparative analysis. Additionally, policy and practice will ultimately never be the same in each country or over time, since it is shaped by the policy environment and the cultural-historic setting. This means that such a study has the second challenge of retaining the information within the analysis that is provided by the variety of contexts and the methods of implementation.

The specific task set for the researchers of the study presented here was formulated as follows: “…to provide a consolidated, accessible and up-to-date overview of the effects of different models of cost-sharing in higher education on participation patterns, the diversity, quality and relevance of educational provision and system efficiency.”

With regard to the specific task of this study then, the third problem of such work is the comprehensiveness of the scope set by the commissioning agency. In this case the object of the study is cost-sharing (i.e. differing the contribution to funding higher education made by the state and private organisations, households or individual persons) and an evaluation of its impacts on higher education institutions' diversity, quality and relevance of educational provision, and on students' participation patterns.

The authors of the study chose the Realist Evaluation approach (Pawson and Tilley 1997, 2004; Pawson 2006) in order to satisfy these challenges and produce a policy-relevant analysis. This approach is designed to improve the understanding of how and why interventions work or do not work in a particular context and has been used frequently in the field of the evaluation of social programmes. As described in a recent paper by the Australian government (Westhorp 2014): “Rather than comparing changes for participants who have undertaken a programme with a group of people who have not, as is done in random control or quasi-experimental designs, a realist evaluation compares whether a programme works differently in different localities (and if so, how and why) or for different population groups (for example, men and women, or groups with differing socio-economic status). Realist evaluations can be undertaken with small or large groups and with qualitative and/or quantitative data.”

The realist evaluation starts out from the programme theory, i.e. the theory about how to bring about a particular change which underpins the specific intervention. This is a common approach in political analysis, since all policy must have an explicit intension at the outset. Following the realist philosophy (Sayer 1992), the approach assumes that social systems are open systems and their boundaries are permeable and flexible. This has important ramifications for any analysis, because it means that the way a particular intervention works (i.e. its outcomes) will vary depending on the context in which it is set. The approach looks to outcomes, but particularly tries to identify “mechanisms” which enable these effects, as it also posits that these mechanisms sometimes fire and sometimes do not (i.e. outcomes may vary). Ultimately, it is people who determine whether this happens based on their decision-making, i.e. it is the interaction between the resources a programme provides, withholds, makes attractive or unattractive and the reaction of intended target group for the intervention which affects the outcome. Importantly, the assumption is that the mechanism will seldom be visible, but must be induced from this interaction. This is an important insight, which is helpful for policy reform, especially as many reforms have historically been found to be ineffective. In this regard, Andrews speaks of successful reforms as positive deviations and sees their openness to interactions and iterative adaptation as one common characteristic for success (Andrews 2013). Following this approach, the task of the researchers is to find or postulate programme theories, which can be investigated in the study.

In the following, the object of study (cost-sharing) will be briefly presented, and then the programme theories associated with it, the research questions and way of collating information for the study follow on from this. Subsequently, the results of the study will be discussed and the paper will close with general considerations for evidence-based policy.

• [1] For instance, the Thematic Review of Tertiary Education carried out by the OECD— oecd.org/edu/skills-beyond-school/thematicreviewoftertiaryeducation.htm