Evidence in Relation to the Effectiveness of E-mediated Personalised Education
Abstract In this chapter, we ponder the challenge of evaluating the effectiveness of personalised learning in e-mediated higher education contexts. We argue that we must first decide what constitutes success. Higher education is a complex learning space, so the goals of teaching and learning will be necessarily complex and diverse. This chapter ends with a final word from the authors. We reflect that we set out to examine the theoretical underpinning of the concept of personalised education and to explore the question: What is e-mediated personalised education in the contemporary higher education sector and how is it enacted? This chapter ends with our insights around this question and a model pulling together the theoretical frames we have borrowed across this book.
What Is Effective?
Davis (2011) notes that, while research is developing within the space of what she describes as “personalised learning”, there are continuing challenges in determining what is most effective, or indeed effective at all. Some of this difficulty arises due to the issues discussed elsewhere in this book in relation to the interests and purposes of education: deciding whether something is effective or not requires a view on the purpose of the activity, since effectiveness is defined as the extent to which something achieves the purposes for which it is intended.
If the goal of e-mediated education is to lift students’ scores on standardised tests, for example, measures of its effectiveness are relatively simple: experimental or quasi-experimental tests comparing gains in the test scores of students studying with or without a particular technology, or comparing competing technologies. A more sophisticated and difficult question, and one just beginning to be attended to in the literature, is the effectiveness of particular pedagogies: something much more difficult to identify and quantify than the presence or absence of a particular technological solution. If the purpose is to enhance student’s confidence and self-efficacy, the measures will be different. If it is to enhance critical thinking
© Springer Nature Singapore Pte Ltd. 2017 B. Garrick et al., Theorising Personalised Education, DOI 10.1007/978-981-10-2700-0_10
skills, the measures will be different again and arguably will need to be more sophisticated.
The really difficult issues, however, arise when either (a) the purposes of the learning activities are not well understood by those developing the innovation or (b) there are competing purposes on the part of different stakeholders.
Another issue complicating assessments of the effectiveness of e-mediated education is what we have referred to elsewhere as technoboosterism:
Much of the published literature in the field of educational technology still tends toward what might be described as ‘technoboosterism’—a relatively uncritical belief that information technology based approaches to teaching and learning will yield improvements in students’ attitude to and engagement with learning as well as in their understanding and achievement. This effect is exacerbated by the fact that often papers are written by the originators of the particular technological application being described, so that many reports are of the ‘I made it, I used it, it was great!’ genre. (Geelan and Mukheijee 2010, pp. 35383539)
Moderating claims made for particular approaches to fit the available, credibly generated evidence is crucial and is, unfortunately, not always current practice. As one example of the excessive claims made with relatively weak or absent evidence, Keefe and Jenkins (2008) were speaking in the context of secondary rather than tertiary education when they entitled their book Personalised Instruction: The Key to Student Achievement. The publisher’s review proclaims “Personalized education can solve most of the instructional issues occurring in North American schools”; however, the work presented is more descriptive of approaches and strategies that the authors have developed and implemented, rather than reporting empirical evidence to support the claims made in relation to achievement.
A number of reviews have critically considered the effectiveness of e-mediated learning (e.g. Clements and Sarama 2003; Cordes and Miller 2000; Kompf 2005; Reeves 1995), but more high-quality, focused research into the effectiveness of these tools for learning—where the purposes of learning are explicit and well understood, and the measures are valid—is still urgently needed if our practice is to be based more on evidence than on pure enthusiasm.
E-mediated learning is a field in which growth is incredibly rapid, and this results both from rapid technological change and from social changes that influence the ways in which technologies are taken up and, perhaps more importantly, the ways in which their effectiveness and efficiency are measured. Studies from the 1990s or even the early 2000s are unlikely to be illuminating in the late “teens” and in future, since the technological, social, and pedagogical landscapes have changed so dramatically in the relatively brief intervening time.
Disambiguating e-mediated education from personalised learning is not always easy in reading reports of research results, since as has been noted their relationship is that of two partially overlapping circles in a Venn diagram rather than being synonyms for one another. Much e-mediated learning, for example, is programmed rather than personalised (see Chap. 6). The challenge is exacerbated by the plethora of different terms used to describe technologies, pedagogies, practices, and commitments that may or may not be the same things.