Is There a Place for Quantitative LOA Research?
So far, this chapter has shown that qualitative research approaches can comprehensively capture the complexity of LOA in L2 classrooms. But what about quantitative research approaches? Regrettably, it is questionable that a purely quantitative study can adequately grasp the nuances of classroom-based LOA. Specifically, it would be a great challenge to properly document how, why, and when learning transpires in planned and unplanned assessment episodes using a purely quantitative approach.
Nevertheless, quantitative data analysis does have a place in classroom- based LOA research. Given the complexity of LOA, some researchers have sought to establish patterns through the quantification of qualitative data. For example, Perrone (2011) observed an EFL classroom, coded his transcribed data into pre-determined categories, and quantified the data by calculating the students’ accuracy of noticing, the accuracy of noticing with repeated practice, speed of noticing, and speed of noticing with repeated practice. The quantification of the data allowed him to coherently compare the performance between a high-achieving student and a low-achieving student. As Mackey and Gass (2005) suggested, quantification can bring several benefits to a qualitative study: it allows patterns in qualitative data to be easily detected, it can facilitate the generation of hypotheses, it helps researchers communicate the findings clearly, and it makes research results more accessible and easily understandable to other researchers.
In conclusion, this chapter examined how LOA research has been conducted in L2 contexts, identified several common data collection and analysis approaches, analyzed the types of evidence in relation to learning that had been provided, and explored the LOA dimensions that have been addressed. Based on the findings, recommendations on how to conduct quality LOA research in L2 classrooms were provided. While LOA is not a new research agenda in L2 contexts, its complexity and multidimensionality always leave more to explore. It is hoped that this chapter provides a starting ground for discussions of how classroom-based LOA can be conducted to comprehensively capture the dynamics of learning, teaching, and assessment in L2 classrooms.
Ali, H. I. H. (2013). In search for implementing learning-oriented assessment in an EFL setting. World Journal of English Language, 3(4), 11-18. doi:10.5430/ wjel.v3n4pl 1
Bailey, A. L., &; Heritage, M. (2014). The role of language learning progressions in improved instruction and assessment of English language learners. TESOL Quarterly, 48(3), 480-506. https://doi.org/10.1002/tesq.176 Baker, B., 5c Germain, J. (2020). Narrative inquiry as praxis: Examining formative assessment practices in a nature-based indigenous language classroom. In M. E. Poehner 5c O. Inbar-Lourie (Eds.), Toward a Reconceptualization of Second Language Classroom Assessment: Praxis and Researcher-Teacher Partnership (pp. 107-127). Springer International Publishing. https://doi. org/10.1007/978-3-030-35081-9_6
Barkhuizen, G., Benson, P., &: Chik, A. (2013). Narrative Inquiry in Language Teaching and Learning Research. Routledge.
Braun, V., 5c Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3(2), 77-101.
Carless, D., Joughin, G., &: Мок, M. (2006) Learning-oriented assessment: Principles and practice. Assessment and Evaluation in Higher Education, 31(4), 395-398. https://doi.org/10.1080/02602930600679043 Chapelle, C. A., Schmidgall, J., Lopez, A., Blood, I., Wain, J., Cho, Y., Hutchison, A., Lee, H.-W., 5c Dursun, A. (2018). Designing a prototype tablet-based learning-oriented assessment for middle school English learners: An evidence-centered design approach (Research Report No. RR-18-46). Educational Testing Service, https://doi.org/10.1002/ets2.12232 Cheng, L., Wu, Y., 5c Liu, X. (2015). Chinese University Students’ Perceptions of Assessment Tasks and Classroom Assessment Environment. Language Testing in Asia, 5(13), 1-17. doi:10.1186/s40468-015-0020-6 Galaczi, E. D. (2014). Content analysis. In The Companion to Language Assessment, A.J. Kunnan (Ed.), doi: 16.1002/9781118411360.wbcla063 Hamp-Lyons, L. (2017). Language assessment literacy for language learning-oriented assessment. Papers in Language Testing and Assessment, 6(1), 88-111. Hill, K., 5c McNamara,T. (2012). Developing a comprehensive, empirically based research framework for classroom-based assessment. Language Testing, 29(3), 395-420. https://doi.org/10.1177/0265532211428317 Jones, N., 5c Saville, N. (2016). Learning Oriented Assessment: A Systemic Approach. Cambridge University Press.
Kim, A.-Y., 5c Kim, H. J. (2017). The effectiveness of instructor feedback for learning-oriented language assessment: Using an integrated reading-to-write task for English for academic purposes. Assessing Writing, 32, 57-71. https:// doi.org/10.1016/j .asw.2016.12.001
Leung, C. (2020). Learning-oriented assessment: More than the chalkface. In M. E. Poehner &: O. Inbar-Lourie (Eds.), Toward a Reconceptualization of Second Language Classroom Assessment: Praxis and Researcher-teacher Partnership (pp. 85-106). Springer International Publishing. https://doi. org/10.1007/978-3-030-35081-9_5
Lincoln, Y., 5c Guba, E. G. (1985). Naturalistic Inquiry. Sage.
Mackey, A., 5c Gass, S. M. (2005). Second Language Research: Methodology and Design. Lawrence Erlbaum Associates Publishers.
May, L., Nakatsuhara, E, Lam, D., 5c Galaczi, E. (2020). Developing tools for learning oriented assessment of interactional competence: Bridging theory and practice. Language Testing, 37(2), 165-188. doi:10.1177/0265532219879044
Mortensen, K., 5c Wagner, J. (2012). Conversation analysis: Overview. In C.A. Chapelle (Ed.), The Encyclopedia of Applied Linguistics. Wiley, doi: 10.1002/978140519843 l.wbeall339
Nakatsuhara, E, May, L., Lam, D., & Galaczi, E. (2018). Learning oriented feedback in the development and assessment of interactional competence. Research Notes, 70, 1-68. Available at: www.carnbridgeenglish.org/Images/517543- research-notes-70.pdf
Navaie, L. A. (2018). The effect of learning-oriented assessment on learning pronunciation among Iranian EFL Learners. International Journal of Education and Literacy Studies, 6(2), 63-68. https://doi.Org/10.7575/aiac.ijels.v.6n.2p.63
Neuendorf, K. A. (2019). Content analysis and thematic analysis. In P. Brough (Ed.), Research Methods for Applied Psychologists: Design, Analysis and Reporting (pp. 211-223). Routledge.
Nowell, L. S., Norris, J. M., White, D. E., 5c Moules, N. J. (2017). Thematic analysis: Striving to meet the trustworthiness criteria. International Journal of Qualitative Methods, 16(1), 1-13. doi:10.1177/1609406917733847.
Perrone, M. (2011). The effect of classroom-based assessment and language processing on the second language acquisition of EFL students. Journal of Adult Education, 40(1), 20-33.
Poehner, M. E., 5c Infante, P. (2017). Mediated development: A Vygotskian approach to transforming second language learner abilities. TESOL Quarterly, 51(2), 332-357. https://doi-org.tc.idm.oclc.org/10.1002/tesq.308
Polkinghorne, D. E. (1995). Narrative configuration in qualitative analysis. Qualitative Studies in Education, 8(2), 5-23. https://doi.org
Purpura, J. E. (2016). Second and foreign language assessment. The Modern Language Journal, 100(SI), 190-208. https://doi.org/10.llll/modl.12308
Purpura, J. E., Banerjee, H. L., Beltran, J., Stabler-Havener, M. (2016, June 20-24). Examining L2 learning and learning outcomes as a result of embedded classroom assessments. Paper presentation 38th Annual Language Testing Research Colloquium, Palermo, Sicily, Italy.
Purpura, J. E., 5c Turner, С. E. (2016). A learning-oriented assessment approach to understanding the complexities of classroom-based language assessment. Paper presentation Roundtable on Learning Oriented-Assessment in Language
Classrooms and Large-Scale Contexts, New York, NY, United States, https:// vvww.tc.columbia.edu/tccrisls/the-tccrisls-program/
Purpura, J. E., 5c Turner, С. E. (2021). Learning-oriented Assessment in Language Classrooms: Using Assessment to Gauge and Promote Language Learning. Routledge.
Scarino, A. (2020). Mediation in the assessment of language learning within an interlingual and intercultural orientation: The role of reciprocal interpretation. In M. E. Poehner 5c O. Inbar-Lourie (Eds.), Toward a Reconceptualization of Second Language Classroom Assessment: Praxis and Researcher-teacher Partnership (pp. 43-60). Springer International Publishing. https://doi. org/10.1007/978-3-030-35081-9_3
Timpe-Laughlin, V., Sc Choi, I. (2017). Exploring the validity of a second language intercultural pragmatics assessment tool. Language Assessment Quarterly, 14(1), 19-35. https://doi.org/10.1080/15434303.2016.1256406 Tobin, G.A.,&Begley,C.M.(2004).Methodologicalrigourwithinaqualitativeframework.
Journal of Advanced Nursing, 48,588-396. doi:10.1111/j.l365-2648.2004.03207.x Tsagari, D. (2014, October 10-12). Unplanned LOA in EFL classrooms: Findings from an empirical study. Paper presentation Roundtable on Learning Oriented- Assessment in Language Classrooms and Large-Scale Contexts, New York, NY, United States, https://www.tc.columbia.edu/tccrisls/the-tccrisls-program/ Turner, С. E. 5c Purpura, J. E. (2016). Learning-oriented assessment in second and foreign language classrooms. In D. Tsagari 5c J. Banerjee (Eds.), Handbook of Second Language Assessment (pp. 255-273). DeGruyter Mouton.
Vaismoradi, M.,Turunen, H., 5c Bondas,T. (2013). Content analysis and thematic analysis: Implications for conducting a qualitative descriptive study. Nursing & Health Sciences, 15(3), 398-405. doi: 10.1111/nhs. 1204 Waring, H.Z. (2019).The what and howof English language teaching: Conversation analytic perspectives. In X. Gao (Ed.), Second Handbook of English Language Teaching (pp. 1053-1070). Springer. doi:10.1007/978-3-319-58542-0_54-l Wolf, M. K., Guzman-Orth, D., Lopez, A., Castellano, K., Himelfarb, I., 5c Tsutagawa, F. S. (2016). Integrating scaffolding strategies into technology- enhanced assessments of English learners: Task types and measurement models. Educational Assessment, 21(3), 157-175. https://doi.org/10.1080/106271 97.2016.1202107
5 Success in Language Learning