This chapter has discussed applications of technology for LOA contexts. The chapter began with the advice that computer technology does not have its own philosophy of teaching, learning, or assessment. Instead, the instructor utilizes technology to support his or her classroom methods. In doing so, it is important to reconsider the constructs that may change with the use of technology and the impact technology use has on learning (Chapelle & Voss, 2016).
Careful consideration must go into the selection of technological tools at the beginning of the planning phase as well as during the analysis of L2 performance of a task using technology. It is important to ask, “Does technology change the construct being measured?” For example, does reading on a screen change the performance of a reading task? Does the screen size affect performance? Choi et al. (2003) compared reading, listening, and grammar tasks on a paper-based test and a computer-based test. They found that the listening and grammar sections had the strongest similarities, while the reading section was the least similar. These findings are not difficult to understand if you think about how a reading task is displayed on a computer screen. It may be difficult to keep track of topics and details in a long reading passage that requires scrolling without the ability to underline or highlight sentences in a text. A second challenge is the limited space to display questions along with a reading passage. Should a passage and the questions be on the same screen? Should questions be to the side or below the reading passage? All of these considerations are important for how students interact with a text and may influence performance on an assessment.
In addition to language assessment considerations such as construct definition, validity reliability, security, and consistent administration, there are infrastructure considerations such as updating hardware and software, having the technical knowledge to develop and maintain the technology, connection speed or blackouts, or even the availability of computer technology, which is often an economic issue.
An additional consideration is the impact or consequences of language assessment through technology, often called washback. Do assessment and instruction use similar technology? To date, not many washback studies have explored the “effect of computer-based tests on how teachers change their instruction style” (Garcia Laborda, 2007, p. 8). Above, we discussed the issue of trying to trick the system by learning how automated scoring algorithms work. Another negative consequence is students using slow, unnatural speech when talking to a computer, thinking that slower speech is more comprehensible. Instructors should be mindful of how students naturally interact with technology and match assessment tasks with familiar computer interactions.
The Role of Technology in Learning-Oriented Assessment 221 Conclusion
This chapter has introduced how technology, hardware, and software can be used to support LOA assessment. It has introduced technological tools to assist in the design of assessment tasks as learning tasks, include students in the assessment process, and provide quality feedback (Carless, 2007) as well as collect and analyze L2 processes during L2 performance (Turner & Purpura, 2015). A deeper understanding of the potential capabilities and challenges of assessing language through computer technology is essential for their appropriate use (Chapelle & Douglas, 2006). Throughout the chapter, reference has been made to how technology use may result in redefining the construct and also the impact new technology methods have on instruction, assessment, and learning. The chapter also included a discussion of innovative, emerging applications of technology that may disrupt current teaching and assessment methods and the privacy concerns that accompany new technology.
Barker, F. (2013). Using corpora to design assessment. In A. J. Kunnan (Ed.). The Companion to Language Assessment. (Vol. 2, Chapter 8). Wiley-Blackwell. https://doi.org/10.1002/9781118411360.wbclal02.
Burstein, J., Frase, L., Ginther, A., Sc Grant, L. (1996). Technologies for language assessment. Annual Review of Applied Linguistics, 16, 240-260. https://doi. org/10.1017/S0267190500001537.
Carless, D. (2007). Learning-oriented assessment: conceptual bases and practical implications. Innovations in Education and Teaching International, 44(1), 57-66. https://doi.org/10.1080/14703290601081332.
Carless, D. (2015). Exploring learning-oriented assessment processes. The Journal of Higher Education, 69,963-976. http://doi.org/10.1007/sl0734-014-9816-z. Carlin, A. S., Hoffman, H. G., 6c Weghorst, S. (1997). Virtual reality and tactile augmentation in the treatment of spider phobia: A case study. Behaviour Research and Therapy, 35, 153-158. https://doi.org/10.1016/S0005-7967(96)00085-X. Chapelle, C. A.,6c Douglas, D. (2006). Assessing language through computer technology. Cambridge University Press. https://doi.org/10.1017/CB09780511733116. Chapelle, C.A., 6c Jamieson, J. (2008). Tips for teaching with CALL: Practical approaches to computer-assisted language learning. Pearson Education. Chapelle, C.A., 6c Voss, E. (2016). Twenty' years of technology and language assessment in language learning & Technology. Language Learning & Technology, 20(2), 116-128. http://dx.doi.org/10125/44464.
Chapelle, C.A., 6c Voss, E. (2017). Utilizing technology in language assessment. In Shohamy, E., 6c Or, I. (Eds.). The Encyclopedia of Language and Education. Language testing and assessment (Vol. 7). Springer Press, https:// doi.org/10.1007/978-3-319-02261 -1_10.
Choi, L-С., Kim, K.S., 6c Boo, J. (2003). Comparability of a paper-based language test and a computer-based language test. Language Testing, 20(3), 295-320. https://doi.org/10.1191/02655322031t258oa.
Chukharev-Hudilainen, E. (2014). Pauses in spontaneous written communication: A keystroke logging study. Journal of Writing Research, 6(1), 61-84. doi: 10.17239/jowr-2014.06.01.3.
Cobb,T. (2020). Web VP Classic v.4 [computer program]. Accessed May 19 2020 at https://www.lextutor.ca/cgi-bin/range/texts/index.pl.
Cotos, E. (2011). Potential of automated writing evaluation feedback. Calico Journal, 28(2), 420-459.
Davies, M. (2011). Word And Phrase (based on data from the COCA corpus). Available online at https://www.wordandphrase.info
Edpuzzle. (2019). Retrieved from https://edpuzzle.com/
Garcia Laborda, J. (2007). On the net: Introducing standardized EFL/ ESL exams, Language Learning & Technology, 11(2), 3-9. http://doi. org/10125/44097.
Gebril, A., 5c Brown, G.T.L. (2020). Quality of feedback in TESOL: A learning- oriented assessment perspective. In J. Agudo (Ed.), Quality in TESOL and teacher education: From results culture to quality culture. Routledge.
Hamari, J., Shernoff, D.J., Rowe, E., Coller, B., Asbell-Clarke, J., 5c Edwards, T. (2015). Challenging games help students learn: An empirical study on engagement, flow and immersion in game-based learning. Computers in Human Behavior, 54,170-179. https://doi.Org/10.1016/j.chb.2015.07.045.
Hoque, M.E. (2013). Computers to help with conversations: Affective framework to enhance human nonverbal skills (Doctoral dissertation). Available from ProQuest Dissertations and Theses database.
Jones, N., 5c Saville, N. (2016). Learning Oriented Assessment: A Systemic Approach: Studies in Language Testing 45. Cambridge University Press.
KAHOOT!. (2019). Retrieved from https://kahoot.com/
Keppell, M. 5c Carless, D. (2006). Learning-oriented assessment: a technology- based case study. Assessment in Education: Principles, Policy & Practice, 13(2), 179-191.
Kessler, G. (2013). Authoring tools for language assessment. In C. A. Chapelle (Ed.), The Encyclopedia of Applied Linguistics. Wilev-Blackwell. https://doi. org/10.1002/9781405198431 ,wbeal0064.
Kol, S., 5c Schcolnik, M. (2008). Asynchronous forums in EAP: Assessment issues. Language Learning & Technology, 12(2), 49-70. http://doi.org/10125/44143.
Leijten, M., 5c Van Waes, L. (2013). Keystroke logging in writing research: Using inputlog to analyze and visualize writing processes. Written Communication, 30(3), 358-392. https://doi.org/10.1177/0741088313491692.
Lloyd, A., Rogerson, S., &: Stead, G. (2017). Imagining the potential for using virtual reality technologies in language learning. In M. Carrier (Ed.) Digital language learning and teaching: Research, theory, and practice, (pp. 222-234). Routledge.
Martln-Gutierrez, J., Efren Mora, C.E., Anorbe-Di'az, B., 5c Gonzalez-Marrero, A. (2017). Virtual technologies trends in education. EURASIA Journal of Mathematics Science and Technology Education, 13(2), 469-486. https://doi. org/10.12973/eurasia.2017.00626a.
Metametrics. (2009). The Lexile framework for reading. Retrieved from http:// www.lexile.com.
Nakatsuhara, E, Inoue C., Berry V, Galaczi E.D. (2017) Exploring the use of video- conferencing technology in the assessment of spoken language: a mixed-methods study, Language Assessment Quarterly, 14(1), 1-18. https://doi.org/10.1080/15 434303.2016.1263637.
Plickers (2019). Retrieved from https://get.plickers.com/
Poehner, M.E., Zhang, J., 5c Lu, X. (2015). Computerized dynamic assessment (C-DA): Diagnosing L2 development according to learner responsiveness to mediation. Language Testing, 32(3), 337-357. https://doi. org/10.1177/0265532214560390.
Purpura, J.E. (2021). Learning-oriented assessment in language classrooms: Using assessment to gauge and promote language learning. Routledge.
Radu, I. (2014). Augmented reality in education: a meta-review and cross-media analysis. Personal and Ubiquitous Computing, 18, 1533-1543. https://doi. org/10.1007/s00779-013-0747-y.
Saville, N. (2013, December). The CFR and Learning Oriented Assessment (LOA) [Conference session], Dubai, Saudi Arabia.
Spelman Miller, K.S. (2005). Second language writing research and pedagogy: A role for computer logging? Computers and Composition, 22(3), 279-317. https://doi.Org/10.1016/j.compcom.2005.05.003.
Suvorov, R. (2015). The use of eye tracking in research on video-based second language (L2) listening assessment: A comparison of context videos and content videos. Language Testing, 32(4), 463^183. https://doi.org/10.1177/0265532214562099.
Suvorov, R. (2018). Test takers' use of visual information in an L2 video-mediated listening test: Evidence from cued retrospective reporting. In G. Ockey 5c E. Wagner (Eds.), Assessing L2 listening: Moving towards authenticity (pp. 145-160). John Benjamins, https://doi.org/10.1075/lllt.50.10suv.
Turner, C.E., 5c Purpura, J. E. (2015). Learning-oriented assessment in the classroom. In D. Tsagari & J. Banerjee (Eds.). Handbook of second language assessment. DeGruyter Mouton.
Van Waes, L., Leijten, M., Van Weijen, D. (2009). Keystroke logging in writing research; Observing writing processes with Inputlog. German as a Foreign Language Journal, 2-3 (pp. 41-64). Retrieved from: http://www.gfl-journal. de/2-2009/vanwaes.pdf.
Voss, E. (2012). A validity argument for score meaning of a computer-based ESL academic collocational ability test based on a corpus-driven approach to test design (Doctoral dissertation). http:/doi.org/10.31274/etd-180810-2398.
Voss, E. (2018). Assessment and Technology. In TESOL Encyclopedia of English Language Teaching. John Wiley 5c Sons, Inc. https://doi. org/10.1002/9781118784235.eelt0388.
Voss, E. (2020). A lesson in hedging with online corpus data in an academic research and writing course. In J.K. Shin 5c P. Vinogradova (Eds.). Contemporary foundations for teaching English as an additional language: Pedagogical approaches and classroom applications. Routledge.
Wagner, E. (2007). Are they watching? Test-taker viewing behavior during an L2 video listening test. Language Learning & Technology, 11(1), 67-86. http:// dx.doi.org/10125/44089.
Wagner, E. (2008). Video listening tests: What are they measuring? Language AssessmentQuarterly,5,218-243. https://doi.org/10.1080/15434300802213015.
Wagner, E. (2010). The effect of the use of video texts on ESL listening test- taker performance. Language Testing, 27(4), 493-510. https://doi. org/10.1177/0265532209355668.
Winke,P., Gass, S., & Sydorenko,T. (2013). Factors influencing the use of captions by foreign language learners: An eye-tracking study. The Modern Language Journal, 97(1), 254-275. https://doi.Org/10.llll/j.1540-4781.2013.01432.x.
Xi, X. (2010). Automated Scoring and Feedback Systems: Where are We and Where are We Heading? Language Testing, 27(3), 291-300. https://doi. org/10.1177/0265532210364643.
Xie, Y., Ryder, L., &c Chan, Y. (2019). Using interactive virtual reality tools in an advanced Chinese language class: A case study. TechTrends, 63, 251-259.
Youglish. (2019). Retrieved from https://youglish.com/
Yuen, S., Yaoyuneyong, G., & Johnson, E. (2011). Augmented reality: An overview and five directions for AR in education. Journal of Educational Technology Development and Exchange, 4(1), 119-140. https://doi.org/10.18785/ jetde.0401.10.