Quantitative Measures

Intercultural Sensitivity Survey

The Intercultural Sensitivity Survey was chosen to address the first research question on the effect of an educator study abroad program on enhancing participants’ intercultural sensitivity. This measure has been used previously to assess the development of intercultural sensitivity in short-term study abroad programs and includes measures of global competency (substantive knowledge, perceptual understanding, and intercultural communication) as well as stages of intercultural sensitivity (denial, defense, minimization, acceptance, adaptation, and integration) (Olson & Kroeger, 2001). It consists of 48 questions self-rated using a 5-point Likert scale as well as several objective questions necessary for obtaining demographic information. Students rated themselves on each item as 1= Never Describes Me, 2= Seldom Describes Me, 3= Describes Me Some of the Time, 4=Describes Me Well, or 5=Describes Me Extremely Well. Each participant scanned the QR code from the instructor to access the survey. The survey was compiled and analyzed in Qualities. This survey was administered during Phases 1, 2, and 3.

Intercultural Knowledge and Competence Value Rubric

The Intercultural Knowledge and Competence Value rubric (Association of American Colleges and Universities, 2009) was chosen to address the second research question on the effect of an educator study abroad program on developing participants’ intercultural knowledge and competence. This measure was developed by teams of faculty experts representing colleges and universities across the United States through a process that examined many existing campus rubrics and related documents for each learning outcome and incorporated additional feedback from faculty (Bennett, 2008). This rubric contains six indicators, Knowledge: cultural self-awareness, Knowledge: cultural worldview framework, Skills: empathy, Skills: verbal and nonverbal education, Attitudes: curiosity, and Attitudes: openness. Each indicator is rated with the following four performance criteria: 1= benchmark, 2 & 3= milestones, and 4= capstone. Faculty rated the participants on a scale of 1—4 as a comprehensive assessment at the end of the project using qualitative measures (e.g., individual daily blogs, weekly debriefing sessions, and direct observations).

Linguafolio

LinguaFolio is an online language learning portfolio that promotes autonomous learning through formative assessment (https://linguafolio.uoregon. edu/). This measure was used to address the third research question, the effect of an educator study abroad program on developing Setswana language and communication skills. This tool enables learners to set goals based on Can-Do Statements, track their progress toward accomplishing the statements, and upload work samples to showcase their abilities. Learners are enabled to truly understand the differing levels of language proficiency through examining and practicing the language functions embodied by the Can-Do Statements. They can create evidence to include in their portfolios on a regular basis to showcase their work, which helps them see how they are building the capacity to use the target language. Learners have a clear voice in tailoring their learning experiences to their own needs, wants, and interests (https://linguafolio.uoregon.edu/). Learners complete activities and set goals using three modules: Biography, Passport, and Dossier. Included in these modules are learning profiles and inventories, and self-assessment checklists to record proficiency levels and ongoing progress with language learning. LinguaFolio Online contains five sections for self-rating on Interpersonal Communication, Interpretive Listening, Interpretive Reading, Presentational Speaking, and Presentational Writing questions. Participants rated themselves on each item as (a) this is a goal, (b) can do with help, (c) can do, or (d) can do well during Phases 1, 2, and 3. The data was compiled and analyzed within LinguaFolio to determine the level of language mastery for each participant.

Qualitative Measures

Debriefing Meetings

Debriefing sessions were held weekly for approximately one hour each session during Phase 2 of the project with both whole and small group discussion. A final debriefing meeting was held upon return to the United States during Phase 3. Topics ranged from reflecting on activities, planning for school visits, airing frustrations and working through team dynamics.

Daily Blogs

All participants reflected on experiences through daily blogs. Participants were creative in designing their own blogs with formats including written journals, online applications, and web pages. Visuals were encouraged by instructors with photos shared daily through a WhatsApp group page.

Observations

Project instructors completed direct observations during school visits, language and content seminars, and daily activities. Instructors worked side-by-side with participants in classrooms and engaged in Setswana language learning. Participant feedback was provided daily during activities.

Results

In Figure 11.1, The Intercultural Sensitivity Survey data show the average scores for all 12 participants for each category, Phase 1 (pre-departure), Phase 2 (in-country), and Phase 3 (post-overseas), calculated for the study abroad course. This measure was selected because the variables relate strongly to interprofessional collaboration. The first six categories on the survey

Figure 11.1 Participant average scores on the Intercultural Sensitivity Survey

represent the stages that an individual goes through to become culturally sensitive; from denial to integration. The greatest score increases from Phase 1 to Phase 3 were in the categories of acceptance (4.3 to 4.6), adaptation (4.0 to 4.4), and integration (3.1 to 3.6).

The next three categories on the survey are measures of global competency and include substantive knowledge, perceptual understanding, and intercul-tural communication. Growth was noted in all three areas with the greatest score increase in intercultural communication from Phase 1 to Phase 2 (3.3 to 3.9) with a slight decrease noted in Phase 3 (3.7). Substantive knowledge was the second area to show a score increase from Phase 1 to Phase 3 (2.6 to 3); and perceptual understanding had a slight increase (4.2 to 4.3).

In Figure 11.2, there were five questions in the survey that showed the greatest increase from Phase 1 to Phase 3. Three of these questions were in the intercultural communication domain:

• • Question 41: I have learned how to produce work with people from other places in the globe.
• • Question 43: I have lived abroad and experienced intense interaction with a variety of people from this other culture.
• • Question 44: I have long-term friendships with several people from other cultures.

Question 18 was in the adaptation domain where participants were asked to rate themselves on I have added to my own cultural skills new verbal and nonverbal communication skills that are appropriate in another culture. Question 26 related to substantive knowledge with participants rating themselves on I have substantive knowledge about at least one other culture outside the United States, and I apply this knowledge with confidence in my professional work.

Top 5 Questions with the Greatest Increase from Pre-Trip to Mid-Point/Post-Trip

Figure 11.2 Top five questions from the Intercultural Sensitivity Survey with the greatest increase across probes

In Figure 11.3, scores from the Intercultural Knowledge and Competence Value Rubric are represented. This measure was completed by the three instructors in Phase 3 as a culminating assessment of each participant in the course and included review of qualitative measures noted above. Participant scores were variable with some participants scoring at the “benchmark” level and others at the “capstone” level. Previous international experience, chronological age, openness, and flexibility may have contributed to these

Intercultural Knowledge and Competence Value Rubric Post-Trip

■ Student 1 ■ Student 2 ■ Student 3 i Student 4 ■ Student 5 ■ Student 6

■ Student 7 ■ Student 8 ■ Student 9 ■ Student 10 Student 11 ■ Student 12

Figure 11.3 Post departure ratings per participant on the Intercultural Knowledge and Competence Value Rubric

Interprofessional Study Abroad Experience 161 results. Six out of the 12 participants demonstrated high scores for openness and empathy, followed by verbal and nonverbal communication, and curiosity. The individual scores for cultural self-awareness and cultural worldview frameworks were more variable.

All 12 participants completed self-evaluations using LinguaFolio. For Phase 1, all participants reported skills in the Novice Low for the five sections of Interpersonal Communication, Interpretive Listening, Interpretive Reading, Presentational Speaking, and Presentational Writing. For Phases 2 and 3, (the results were the same for both time points) with skills increasing into the Novice Mid and Novice High levels of performance on all five sections. In the Interpersonal Communication section, an example of a novice low question was “I can tell someone my name”, a novice mid question was “I can talk about places I know”, and a novice high question was “I can ask about and identify familiar things in a picture”.