Designing learning with semiotic technologies

Semiotic technologies are tools that are used in particular ways as part of social practices. From a social semiotic perspective, semiotic technologies can be used by the teacher to represent knowledge, define pedagogic relations, and contribute to the organisation of the learning experience for students. Inherent to a semiotic technology are its affordances. The use of a semiotic technology will commit the teacher to the nature of the pedagogy enacted and influence the design of the students’ learning environment and their experience.

There are common and popular semiotic technologies, both digital and traditional, that are now present in the classrooms of most urban schools. These include the whiteboard,screen, and projectors, as well as computing devices used by teachers and sometimes students. This chapter hopes to provoke teachers to reflect on the gains and losses in the use of these semiotic technologies, and in a more productive enterprise, to consider how the affordances of these resources can be harnessed to design for meaningful learning experiences.

Technologies for learning are advancing rapidly, with new tools and applications fast emerging that promise novel ways of teaching and learning in the classroom. These include the use of social media, analytics, augmented and virtual reality, immersive worlds, as well as forms of robot(ic)s harnessing artificial intelligence for educational purposes. The growing list is expected to get more interesting and exciting with time. While the material technological products will continue to change, the theoretical perspective of understanding these resources as semiotic technologies can help us have a consistent set of lenses in understanding the types of metafunctional meanings that these semiotic modes offer as well as reflect on what they can or cannot do. This chapter discusses the value of understanding these new and emerging digital tools as semiotic technologies for learning, which can be harnessed, or resisted, in the classroom, based on their pedagogic value.The ultimate aim is to invite the teacher to think about the range of semiotic technologies she has at her disposal, and what she can bring into the classroom, in her role as a learning designer.

REFLECTION QUESTIONS

• 1. What are the ways in which a semiotic technology used can facilitate the expression of knowledge?
• 2. How does a semiotic technology used shape the pedagogic relationship between the teacher and students?

References

Beauchamp, G. & Parkinson.J. (2005). Beyond the wow factor: Developing interactivity with the interactive whiteboard. School Science Review 86(316), 97—103.

Bebell, D. & O’Dwyer, L. M. (2010). Educational outcomes and research from 1:1 computing settings. Journal ofTcchnology, Learning and Assessment 9(1), 4—15.

Chew, E.C. & Mitchell, A. (2019). Bringing art to life: Examining poetic gameplay devices in interactive life stories. Games and Culture, 1—28.

Christie, E (2002). Classroom Discourse Analysis: A Functional Perspective. London & New York: Continuum.

Cope, B. & Kalantzis, M. (2015).The things you do to know: An introduction to the pedagogy of multiliteracies. In В. Cope & M. Kalantzis (eds), A Pedagogy of Multiliteracies: Learning by Design (pp. 1—36). London: Palgrave Macmillan.

Dickinson, D.K., Hirsh-Pasek, K.,Golinkoff, R.M., Nicolopoulou.A., & Collins, M. F. (2013, April 19).The Read-Play-Learn intervention and research design. Paper presented at the biennial meeting of the Society for Research in Child Development in Seattle. WA.

Djonov, E. & van Leeuwen.T. (2011).The semiotics of texture: From tactile to visual. Visual Communication 10(4), 541-564.

Djonov, E. & van Leeuwen.T. (2012). Normativity and software: A multimodal social semiotic approach. In S. Norris (ed.). Multimodality and Practice: Investigating Theory-in-Practice-through-Method (pp. 119-137). New York: Routledge.

Djonov, E. & van Leeuwen, T. (2013). Between the grid and composition: Layout in PowerPoint’s design and use. Semiotica 197,1-34.

Djonov, E. & van Leeuwen.T. (2014). Bullet points, new writing, and the marketization of public discourse: A critical multimodal perspective. In E. Djonov & S. Zhao (eds). Critical Multimodal Studies of Popular Discourse (pp. 232—250). New York and London: Routledge.

Djonov, E. & van Leeuwen.T. (In press). Semiotic software through the lens of systemic functional theory. In J.R. Martin, J.S. Knox, & D. Caldwell (eds) (In press) Appliable Linguistics and Social Semiotics: Developing Theory from Practice. London: Bloomsbury Academic.

Dostal.J. (2011). Reflections on the use of interactive whiteboards in instruction in international context. The New Educational Review 25(3), 205—220.

Gibson, J. (1966). The Senses Considered as Perceptual Systems. Santa Barbara, CA: Praeger.

Greenberg, D (2000). Avoiding Death by PowerPoint: 45 Proven Strategies to Breathe Life into Dull Presentations. Georgia: Goldleaf Publications.

Hirsh-Pasek. K.. Zosh, J.M.. Michnick, G., Gray, J.H., Robb, M.B.. & Kaufman, J. (2015). Putting education in 'educational' apps: Lessons from the science of learning. Psychological Science in the Public Interest 16(1), 3—34.

Kress, G. (2003). Literacy in the New Media Age. London: RoutledgeFalmer.

Kress, G. (2005). Gains and losses: New forms of texts, knowledge and learning. Computers and Composition 22(1), 5-22. https://doi.Org/10.1016/j.compcom.2004.12.004

Kress, G. (2010). Multimodality - A Social Semiotic Approach to Contemporary Communication. London & New York: Routledge.

Kress, G. (2015). Gunther Kress. In Т.Н. Andersen, M. Boeriis, E. Maagera, & E. Seip Tannessen (eds), Social Semiotics: Key Figures, New Directions (pp. 69—92). London: Routledge.

Kucirkova, N. (2019). Children’s agency by design: Design parameters for personalization in story-making apps. International Journal of Child-Computer Interaction , 21, 112-120. https://doi.Org/10.1016/j.ijcci.2019.06.003.

Lemke, J.L. (1984). Semiotics and Education. Monograph in Toronto Semiotic Circle Monographs Series.Victoria University,Toronto.

Lim, F.V. & Phua, J. (2019).Teaching writing with language feedback technology. Computers and Composition, 54,1—13.

Lim, F.V. & Toh.W. (submitted for publication a). Digital Play for Learning: A Multimodal Discourse Analysis Approach to Educational Apps.

Lim, F.V.,& Toh.W. (submitted for publication b).What is a ‘Good’ Educational App? A Social Semiotic Approach to Evaluating Educational Apps.

Lindstrom, C. (2010). Link Maps and Map Meetings: A Theoretical and Experimental Case for Stronger Scaffolding in First Year University Physics Education (unpublished doctoral dissertation). University of Sydney, Australia. Retrieved from https://Sydney. edu.au/science/physics/pdfs/research/super/Christine_Lindstroms_PhD_thesis.pdf (accessed: 26. Sep 2019).

Lindstrom, C. & Sharma, M.D. (2009). Using link maps to navigate through the physics landscape. Higher Education Research and Development Society of Australasia 31(3), 15-17.

Long, P. & Siemens, G. (2011). Penetrating the fog: Analytics in learning and education. Educause Review. Retrieved from https://er.educause.edu/articles/2011/9/penetrating-the-fog-analytics-in-learning-and-education (accessed: 25 Sep. 2019).

NCTE (2013). NCTE position statement on machine scoring |Web log post] Apr. 20. Retrieved from www2.ncte.org/statement/machine_scoring/ (accessed: 25 Sep. 2019)

Nicolopoulou, A., Toub, T.S., Hassinger-Das, B., Nesbitt, K.T., Ilgaz, H., Dickinson, D.K., Weisberg, D.S., Hirsh-Pasek, K., & GolinkofF, R.M. (2018). The language of play: Developing preschool vocabulary through play following shared book-reading. Early Childhood Research Quarterly 45,1—17.

Norman, D. (1988). The Design of Everyday Things. New York: Basic Books.

OECD (2015). Students, Computers and Learning: Making the Connection. Pisa: OECD Publishing.

Paul, R. & Elder, L. (April 1997). Foundation for Critical Thinking. Available from www. criticalthinking.org (accessed: 25 Sep. 2019).

Poulsen, VS., К vale, G., & van Leeuwen.T (2018). Special issue: Social media as semiotic technology. Social Semiotics 28(5), 593-600.

Prinsloo, P. & Slade, S. (2013). An evaluation of policy frameworks for addressing ethical considerations in learning analytics. Proceedings of the Third International Conference on Learning Analytics and Knowledge - LAK 13. doi: 10.1145/2460296.2460344

Rosen, Y. & Beck-Hill, D. (2012). Intertwining digital content and a one-to-one laptop environment in teaching and learning: Lessons from the Time to Know Program. Journal of Research on Technology in Education 44,225-241.

Rosenthal, R.&Jacobson, L. (1968). Pygmalion in the Classroom: Teacher Expectation and Pupils’ Intellectual Development. Bancyfelin, Carmarthen, Wales: Crown House Pub (1992 new edition).

Selander, S., Lim, F.V., Wiklund, M., & Fors, U. (2018). Digital games and simulations for learning. In H.C. Arnseth, T. Hanghoj, T.D. Henriksen, M. Misfeldt, S. Selander, & R. Ramberg (eds), Games and Education: Designs in and for Learning (25-35). Rotterdam: Sense Publishers.

Shapley, K., Sheehan, D., Maloney, C., & Caranikas-Walker, F. (2011). Effects of technology immersion on middle school students’ learning opportunities and achievement.Journal of Educational Research 104,299-315.

Sinclair, J. & Coulthard, M. (1975). Towards an Analysis of Discourse: The English Used by Teachers and Pupils. London: Oxford University Press.

Smith, H.J., Higgins, S.,Wall, K., & Miller,J. (2005). Interactive whiteboards: Boon or bandwagon? A critical review of the literature. Journal of Computer Assisted Learning 21(2), 91-101.

Tan, J.P.L., Koh, E., Jonathan, C., & Yang, S. (2017) Learner dashboards a double-edged sword? Students’ sense-making of a collaborative critical reading and learning analytics environment for fostering 21st century literacies. Journal of Learning Analytics 4(1), 117— 140. doi: 10.18608/jla.2017.41.7

van Leeuwen, T, Djonov, E., & O'Halloran, K.L. (2013). ‘David Byrne really does love PowerPoint’: Art as research on semiotics and semiotic technology. Social Semiotics 23(3), 409-423.

Wang, Y., Chen, J.. & Yue, Z. (2017). Positive emotion facilitates cognitive flexibility: An fMRI study. Frontiers in Psychology, 8(1832). doi: 10.3389/fpsyg.2017.01832. Retrieved from www.ncbi.nlm.nih.gov/pmc/articles/PMC5671657/

Zhao, S. & van Leeuwen, T. (2014). Understanding semiotic technolog)' in university classrooms: A social semiotic approach to PowerPoint-assisted cultural studies lectures. Classroom Discourse 5(1), 71—90.

Zhao, S., Djonov, E., & van Leeuwen,T. (2014). Semiotic technology and practice: A multimodal social semiotic approach to PowerPoint. Text &Talk 34(3), 349—375.

Zheng, B.,Warschauer, M., Lin, С.-H., & Chang, C. (2016). Learning in one-to-one laptop environments: A meta-analysis and research synthesis. Review of Educational Research 86, 1052-1084.