Participatory Evaluation and Empowerment Evaluation

Participatory approaches to evaluation attempt to privilege the voices of participants and beneficiaries in the assessment of the outcomes of the intervention. Based on the philosophical principles of constructivism, participatory and empowerment evaluation models view outcomes as flexible, critical and situated since they are mainly identified and defined by the users and beneficiaries themselves rather than the evaluator. Accordingly, the focus is on the participants who are best able to identify their needs and whose insights are invaluable in defining the problem, in designing the intervention and also ascertaining whether the outcomes are successful (Chambers, 2009; Hall & Arvidson, 2014; Nichols, 2002). In the case of marginalized groups and the disempowered, participatory action research activities offer them the opportunity to gain more control of their lives and to empower them to be actively involved in enacting and sustaining the change that is required to improve their standard of living. Both participatory evaluation (PE) and empowerment evaluation (EE) share the same commitment to societal change and capacity building. For EE however, the emphasis is on achieving goals of social justice for ‘disenfranchised minority groups’, while practical PE is defined by the involvement of stakeholders in a partnership with facilitators in the design of the evaluation and who then ultimately share the responsibility for the development of the evaluation report (Smits & Champagne, 2008). Rather than project managers having to be only accountable to the funders and those who hold the purse strings, with participatory evaluation, the beneficiaries are afforded the ownership as the persons to whom the SE managers are really accountable to in the overall evaluation process.

At the theoretical level, EE and PE propose that in the collaboration of participants and evaluators, there is the co-creation of new knowledge that encourages the instrumental use of the findings and results which in turn becomes ‘actionable knowledge’ that addresses the problem that was the focus of the intervention (Smits & Champagne, 2008). Within the developmental context, PP and EE have been preferred as they seem to shift the preoccupation with measuring impacts to the notion of managing for sustained impacts that lead to real societal change (Ofir, 2013). As alternatives to the positivist evaluation methodologies, participatory evaluation claims to engage in evaluation for development rather than merely only assessing the characteristics of the developmental process. With the emphasis of participatory methods on capacity building, co-creation of knowledge and organizational learning, there is the opportunity to relate these outcomes to specific change programmes and activities on the ground that provide some evidence of the pragmatic legitimacy of the intervention. By applying the evaluation process as a mechanism for development, participatory methods appear to be much more equipped to tackle poverty reduction, income generation and unemployment which are indicators of social impacts. The evidence of success of participatory models are therefore demonstrated in change of behavior and attitudes where individuals or small community groups are empowered to act to compete for resources, influence policy making and are networked to others outside their groups to access resources and engage in productive exercises where previously this was not the case (Miller & Campbell, 2006).

Among social services and rehabilitative health programs, participatory evaluation methods have been widely advocated as they provide the means for participants to be involved in the design of the change program and to monitor and self- assess the recovery journey in the overall strategy for personal change. An example of this PE method is the Outcomes Star (OS) model that has been developed as a tool to assess the effectiveness of reform and rehabilitation programs targeting a range of social issues such as homelessness, mental health and drug recovery. According to Hall and Arvidson (2014) as it has been developed as a holistic model to be integrated into the working activities of the organization, the OS model is operationalized as a service rather than a separate evaluation exercise. The aim of the OS model is to not only to measure the outcomes of the interventions, but to also provide guidance in achieving the desired outcomes. In its application, the OS model is based on a scale of expected behavior represented as a star that maps out a model of change indicating the steps that are to be undertaken to gain the desired outcomes are that the users are hoping to achieve. In an overall interactive process, the user is encouraged to reflect on past actions and in the process make determinations on the relationship between behavior and outcomes. In this way, the user owns the evaluation process and is enabled to assess and then to make decisions on future pathways for change. But the activity of self-assessment that involves subjective judgments and feelings of the users poses problems of the accuracy or reliability of these accounts. Self-reporting methods have shown that participants may not be totally truthful in these exercises and tend to present positive reports and give information that they think the evaluator will like to hear (Hall & Arvidson, 2014:149). In such instances, it is difficult to control for bias and the trust between users and the facilitator may be broken down if there is requirement to provide verification such as mandatory blood tests as for example in the case of a drug reform program (White, 2013). An important caveat here is that even though the underlying tenets of participatory evaluation do not focus on being objective and value free, they still adhere to the fundamental principles of being evidence based. This means that constructivist knowledge production are also expected to be verifiable and based on empirical data gathering and analysis that clarify the outcomes and impacts of interventions.

The extent to which participatory evaluation methods have been able to provide the empirical evidence to support the claims of empowerment and social impacts have been a major contention. The lack of case study evidence, unanimity in practice as well as the fact that both PE and EE bear similar attributes to general change and social justice theories; all contribute to the blurring of the distinctive contribution of participatory evaluation in theory and practice. The theoretical gaps emerge at moments of operationalization of the participatory methods in terms of clearly defining the context where such methods are suitable and the exact role of the evaluator in facilitating the transfer of power to participants in the evaluation process (Miller & Campbell, 2006). The major risk to participatory evaluation methods is that they may become so normalized that they function more as a rhetorical set piece for development interventions that function essentially as an ideal type rather than a practical program of change.

The key, defining purpose of participatory development is the engagement of a bottom-up process that enables participants to build skills and competencies that allow for recognition and access to productive resources as well as to influence policymaking and governance. Consequently there is a need for participatory methods to interrogate the context of the parameters of social change in terms of the realities of the external environment where these interventions are situated. While PE and EE activities may educate, equip and train individuals and small community groups; the possibilities to enact change will still depend on the external regulatory and political framework. Societal change involves the negotiation of power between those who are in control and those who wish to gain control. Development is not solely a function of the enhancement of the skills sets and market potential of marginalized groups, but also requires external validation and support. As Scarlato (2013) contends, the participatory activities of social projects in many developing nations have yet to address ‘the mechanism through which poverty persists and is embedded in and reproduced by social relations inside specific groups and territories’. Undoubtedly, participatory evaluation methods provide some space for marginalized groups to tackle the problem of social exclusion and to be more proactive agents of change, but it should be also acknowledged that the predominant determinants of change are still measured by principles of new performance management characterized by principles of accountability based on economic efficiency and effectiveness (Chouinard, 2013; White, 2009). As such, the next generation of participatory evaluation methods must move towards an engagement in an agenda of social mobilization in order to attain the credence and persuasive power to actualize claims of ‘societal change’.