Case Study Evaluation Approach

In conducting program evaluation, case study evaluation offers distinct advantages as compared with other approaches. As Yin (2017) explains, “although other methods may assess the outcomes of an intervention or initiative, case study research has a strong advantage in examining the relevant processes" (p. 322). Over the past 20 years, a rigorous methodology has been developed to guide the case study approach to evaluation, particularly in the work of Robert Stake and Robert Yin. In alignment with the recommendations from both Stake (1995) and Yin (2017), this singlecase design incorporates holistic analysis that includes both the context and the case itself. The scope of program implementation bounds the case and the broader history, activities, and structures of the university provide the context.

Our case study evaluation involves substantial experiential knowledge of the case, more in line with Stake’s (1995) approach to emergent case study design than Yin’s (2017) focus on preordinate design. Yet, our approach maintains a commitment to the rigorous, multimcthod inquiry recommended by Yin (2017). In this study, the multiple methods of data collection include review of all relevant program documents, review of the strategic plan materials relevant to the program and related projects, review of feedback forms from all stakeholders, and primary survey data collection of faculty/staff perceptions of their own vocation and CYTI involvement. In addition, the analysis presented here follows the methodological

Re-membering the Mission 37 recommendations to organize the analysis around a prescriptive framework or logic model (Yin, 2017).

Participants

Faculty' and staff members were specifically and purposefully selected to participate in CYTI based on their leadership in community engagement. As might be expected in real-world implementation, selection for participation in CYTI—while purposefill—did not proceed from some clearly established and explicit selection criteria. Since implementation science suggests that staff selection is a key competency driver (Fixsen, Naoom, Blase, Friedman, & Wallace, 2005), the case study analysis includes comparative analysis of faculty/staff involved with CYTI as compared to faculty/ staff with no involvement in CYTI. Evaluating the fidelity of the program implementation should include demonstrating appropriate dispositional profiles of program leadership regarding vocation, leadership, and community' engagement.

Instrument and Procedure

A survey was sent to all 34 individuals who served as faculty and staff support for the first years of the CYTI initiative. The survey was composed of nine items for all respondents, as well as an additional five items for CYTI participants. Survey respondents indicated their level of agreement with each item using a four-point, forced-choice Likert scale. The survey was distributed electronically using Qualtrics. Twenty' (m = 20) of the 34 faculty and staff involved in CYTI completed surveys, for a response rate of 58.8%. To form a comparison group, 35 faculty and staff who were not involved in CYTI were also sent the survey. These individuals were selected at random using the faculty/staff directory' as a sampling frame and a random number generator for selection. With this group, only ten (» = 10) submitted complete surveys, for a response rate of 28.5%.

 
Source
< Prev   CONTENTS   Source   Next >