Culturally Responsive Evaluation and the School Counseling Program

In the US, culture is a part of our educational programs and services. For example, in school districts across the US, several different languages are spoken among families and English is the second language or not spoken at all. The different languages indicate that there are different cultures present in the school community. Thus, culture is a dynamic that a school counselor must learn to navigate as they work to provide and evaluate the school counseling program and services.

Recognizing that any evaluation of educational programs in the US requires skills and abilities to navigate the cultural dynamics of the community, the American Evaluation Association developed competencies for evaluators. Referred to as the Statement on Cultural Competence in Evaluation (American

Evaluation Association, 2011), the document asserts the importance of recognizing the presence of culture in evaluation and, for those that evaluate programs, to find ways to develop and implement what is referred to as culturally responsive evaluation.

Interestingly for school counselors, the fields of counseling, psychology, and healthcare have done considerable work over the last 30+ years in recognizing the importance of culture to these professional practices. As a discipline and field, evaluation has learned from the work of the mental health fields to inform the Statement on Cultural Competence in Evaluation and to implore the evaluation community to work toward culturally responsive evaluation (Trevisan & Carey, 2019).

A strong recommendation in the literature on culturally responsive evaluation is to engage stakeholders in the evaluation. We will further discuss the importance of culture and ways to address culture in school counseling evaluation in Chapter 3, the chapter on the importance of stakeholder engagement. We will also provide recommendations on culturally responsive evaluation in the other chapters focused on the components of the school counseling evaluation framework.

Summary

This chapter developed the concept of evaluation by providing a rigorously constructed definition of evaluation, a definition that was constructed with broad input from many different professional organizations, including the АСА; distinguished evaluation from social science research; and detailed the purposes and types of evaluation relevant for school counselors. Further, an evaluation framework was provided that forms the basis for the evaluation work of school counselors and is the major focus of this book. The chapter made an initial connection between the disciplines and fields of school counseling and evaluation. Content highlights include

• • Purposes for evaluation that include program improvement and assessment of impact.
• • Formative evaluation and summative evaluation to address the above, respectively.
• • Distinguishing features of evaluation that differentiate it from social science research which include stakeholder engagement and a focus on providing information for decision-making about a program or policy.
• • An evaluation framework that includes (1) stakeholder involvement, (2) the development of a theory of action or logic model, (3) evaluation questions, (4) evaluation design and methods, (5) data analysis and findings, and (6) communication and use of the evaluation results.
• • Standards that signal quality in evaluation work.
• • The importance of conducting culturally responsive evaluation.

References

Alkin, M. C. (2013). Context-specific evaluation. In M. C. Alkin (Ed.), Evaluation roots: A wider perspective of theorists’ views and influences (pp. 283-292). Los Angeles, CA: Sage Publications.

American Evaluation Association. (2011). Public statement on cultural competence in evaluation. Fairhaven, MA: Retrieved from www.eval.org

American School Counselor Association. (2012). TheASCA national model: A framework for school counseling programs (3rd ed.). Alexandria, VA: Author.

Carey, J. & Trevisan, M.S. (2017, October). Evaluation in support of school counseling in Italian schools. Workshop Presented at Pratiche Educative, Di Cura E Benessere (Counseling in Action), Annual Meeting of Continuando a Crescere, Verona, Italy.

Christie, C. A. (2007). Reported influence of evaluation data on decision makers’ actions: An empirical examination. American Journal of Evaluation, 28(1), 8-25.

Donaldson, S. I., Cooler, L. E., & Scriven, M. (2002). Strategies for managing evaluation anxiety: Toward a psychology of program evaluation. American Journal of Evaluation, 23(3), 261-273.

Donaldson, S. I. & Lipsey, M. W. (2006). Roles for theory in contemporary evaluation practice: Developing practical knowledge. In I. F. Shaw, J. C. Green, & M. M. Mark (Eds.), The Sage handbook of evaluation (pp. 56-75). Thousand Oaks, CA: Sage.

Lee, S. M. & Trevisan, M. S. (2019). Evaluation in support of school counseling in Korea: A proposal. Korean Journal of Education policy, 16(1), 63-80.

Martin, I. & Carey, J. (2014). Development of a logic model to guide evaluations of the ASCA National Model for school counseling programs. The Professional Counselor, 4(5), 455-466.

Mathison, S. (2008). What is the difference between evaluation and research - and why do we care? In N. L. Smith & P. R. Brandon (Eds.), Fundamental issues in evaluation (pp. 183-196). New York, NY: Guilford Press.

Owen, J. M. & Rogers, P. ). (2007). Program evaluation: Forms and approaches. London, UK: Sage.

Preskill, H. & Jones, N. (2009). A practical guide for engaging stakeholders in developing evaluation questions. Robert Wood Johnson Foundation Evaluation Series. Retrieved from www.rwjf.org

Scriven, M. (1967). The methodology of evaluation. In R. E. Stake (Ed.), Perspectives on curriculum evaluation (American Educational Research Association monograph series on evaluation, No. 1). Chicago: Rand McNally.

Scriven, M. (2008). The concept ofa transdiscipline: And evaluation as a transdiscipline. Journal of Multidisciplinary Evaluation, 5(10), 65-66.

Stufflebeam, D. L. & Coryn, C. L. S. (2014). Evaluation theory, models, and applications (2nd ed.). San Francisco, CA: Jossey-Bass.

Trevisan. M.S. & Carey, J. (2017, March). Program Evaluation for School Counselors. Workshop presentation at the Fifth Annual Evidence-Based School Counselors National Conference, San Diego, CA.

Trevisan, M.S. & Carey, J.C. (2019). Evaluating intercultural programs and interventions. In A. Portera, R. Moodley, & M. Milani (Eds.), Intercultural mediation, counselling and psychotherapy in Europe. Cambridge Scholars, UK. Edited book under review.

Trevisan. M. S. & Carey, J. C. (in press). Evaluation in support of counseling in Italian schools. Revista Italiana di Counseling.

Walser, T. M. & Trevisan, M. S. (2019). Completing your evaluation dissertation, thesis, or final project. Los Angeles, CA: Sage. Manuscript under review.

Weiss, С. H. (1988). Evaluation for decisions: Is anybody there? Does anybody care? Evaluation Practice, 9(1), 5-19.

Weiss, С. H. (1995). Nothing as practical as good theory: Exploring theory-based evaluation for comprehensive community initiatives for children and families. In J. P. Connell, A. C. Kubisch, L. B. Schorr, & С. H. Weiss (Eds.), New approaches to evaluating community initiatives: Concepts, methods, and contexts (pp. (>5-92) Washington, DC: Aspen Institute.

Weiss, С. H. (1998). Have we learned anything new about the use of evaluation? American Journal of Evaluation, 19(1), 21-33.

Yarbrough, D. B., Shulha, L. M.. Hopson, R. K., & Caruthers, F. A. (2010). The program evaluation standards: A guide for evaluators and evaluation users (3rd ed.). Thousand Oaks, CA: Corwin Press.