Evaluating Evidence for Video Game Transfer

Digital games have been hypothesized to offer a number of potential instructional benefits as a learning medium (O’Neil, Wainess, & Baker, 2005), including interactivity, which outside the world of video games has been associated with deeper understanding and more successful far transfer (Barnett & Ceci, 2002; Bransford, Brown, & Cocking, 1999; Reed & Saavedra, 1986; Halpern, 1998). However, “while effectiveness of game environments can be documented in terms of intensity and longevity of engagement (participants voting with their money or time), as well as the commercial success of the games, there is much less solid information about what outcomes are systematically achieved by the use of individual and multiplayer games to train participants in acquiring knowledge and skills____What is

missing is how games should be evaluated for education and training purposes” (O’Neil et al., 2005, p. 456). To attain this goal, it’s been suggested that assessment be built into the learning games themselves: “Games that teach also need to be games that test,” (Michael & Chen, 2005). However, such near transfer tests may not necessarily translate to successful far transfer to contexts outside video games, depending on the particular goal of the training program.

Even when the transfer goal is clear, that is, when the purpose is to train employees for particular work-related tasks, transfer testing cannot simply focus on that set of tasks in the work environment, without bearing in mind that a demonstration of successful transfer cannot necessarily be generalized to different training and transfer situations. That is, if a pilot test of a particular training program, which shows successful transfer, is conducted in an unrealistic environment where the transfer required is only to near contexts (testing soon after training in the same location by the same individuals), the findings may not generalize to real world applications of the same training program when scaled up. For example, Rosser, Lynch, Cuddihy, Gentile, Klonsky, and Merrell (2007) studied the relationship between surgeons’ video gaming experience and performance on a laparoscopic surgery skills game. They found a significant relationship. However, as pointed out by Curet (2007), scores on the laparoscopic surgery skills game did not necessarily relate to skill at actual surgery. Perhaps, for example, surgeons performing under the stress of a real life on the line might behave differently from when they are merely playing (that is, in a different functional context).

The problem of how to assess transfer is even more complex when assessing the benefits of games for other educational purposes, because the desired outcomes may be less clear. In both cases, research on transfer of learning from video games can be evaluated by situating findings in the taxonomy of transfer content and context described above. One area in which video game play has been found to improve performance in transfer tasks for a potentially important skill is in three-dimensional (3-D) mental rotation: the visualization and imaginary rotation of an object that is presented as a two-dimensional drawing. This concept was introduced to the field by Shepard and Metzler in 1971 and further explored by Vandenberg and Kuse (1978), who popularized the classic mental rotation test. Each stimulus in this test is a two-dimensional image of a 3-D object. Each object is shown at different orientations and participants are required to recognize, as quickly as possible, which images represent rotated versions of the same object. Reliable gender differences are found on this measure, in favor of males.

The male superiority on tests of 3-D mental rotation has been the subject of a great deal of discussion in the debate surrounding the disproportionate number of men at the top of science, technology, engineering, and mathematics (STEM) fields (see Ceci, Williams, & Barnett, 2009). The overrepresentation of men in these fields has been attributed by some (see Summers, 2005) to innately superior mathematical skills. Although males were once thought to do better, on average, than females on all aspects of mathematics, in the face of more recent evidence, the supposed area of superiority has been narrowed to spatial skills only, and even more recently narrowed again to the particular skill of 3-D mental rotation, a skill for which there is reasonably robust evidence of superior average male performance. Whether this skill is linked to the overrepresentation of men in STEM jobs is currently unknown, but the finding has led to increased interest in understanding causes of differences in 3-D mental rotation skills. The argument for an innate difference in ability between males and females has been bolstered by findings of gender differences among kindergartners (Casey, Andrews, Schindler, Kersh, Samper, & Copley, 2008). However, evidence from innovative video game training studies (Terlecki & Newcombe, 2005; Feng, Spence, & Pratt, 2007) argues in favor of an experience-based explanation. If 3-D mental rotation may be a factor limiting the advancement of women in STEM fields, it is important to understand whether video game play can improve 3-D mental rotation skills in a durable way that transfers.

Feng, Spence, and Pratt (2007) trained participants by having them play a 3-D first-person shooter game (Medal of Honor) for several sessions in their laboratory. Such games require intense visual monitoring and attention. The hypothesis explored was whether spatial attention distribution, a “basic capacity that supports higher-level spatial cognition” (p. 850), could be modified by playing a video game and whether improving individuals’ spatial attention distribution would also lead to improved higher level mental rotation ability (3-D mental rotation). The control group played a different 3-D computer game (a maze puzzle game) that did not involve focused attention on a target and was therefore not expected to improve distribution of spatial attention. Spatial attention was assessed using the Uniform Field of View task, in which participants are required to indicate the direction in which a target has very briefly appeared, after a visual mask. Mental rotation ability was assessed using Vanderberg and Kuse’s (1978) test, described above, in which different 2-D representations of a 3-D object must be recognized as representing the same object. Results confirmed the hypothesis and showed a reduction in the preexisting gender difference on both measures.

So does this mean that video game play can solve the problem of the dearth of women at the top of STEM fields? Clearly, the answer partly depends on the relevance of these skills to the success of women—compared to the relevance of other possible skill differences and societal and discriminatory factors—which learning studies such as that by Feng et al. (2007) do not address. It also depends on whether improvements shown in these kinds of studies robustly transfer. In general, the further the contexts to which transfer is demonstrated in the experimental situation (testing in a physical context remote from the context of learning; learning and transfer tasks differing in purpose and in contrasting modalities; testing after a substantial time has passed since learning occurred; etc.), the further we can be confident the skills will transfer outside of the experimental situation.

As can be seen in Table 2.3, apart from the knowledge domain, all aspects of context for the Feng, Spence, and Pratt (2007) posttest were near to the context of training. Assessments were conducted soon after training, in the same lab, and both training and transfer tests were overtly research-oriented, involved working individually, and were computer-based. However, the knowledge domain, involving transfer from shooting virtual soldiers to locating the direction of dots on a screen and mentally rotating abstract shapes, can be considered somewhat far transfer. Also, the follow-up test assessed durability of the enhancement an impressive five months later.

Presumably, transfer of these skills would be desirable in many contexts: in book and lab work; at school, in research institutions or in the workplace; in test performance; and at a later time, months or even years later. Thus, for example, it is important to know whether similar improvements would have been found if the training and transfer tasks had not both been computer-based. Would a team of engineers designing a bridge be better at visualizing their plans and detecting design issues from various perspectives if they were trained on Medal of Honor? Would a study group have enhanced success on their trigonometry exam? What is the likelihood that improving these skills will have an effect on women’s success in STEM fields? Greene, Li, and Bavelier (2009) have suggested that action video game experience teaches individuals to “form templates for, or extract the relevant statistics of, the task at hand” (p. 1). What we lack is an understanding of how and when aspects of STEM jobs might tap into such skills.

Table 2.3 Transfer Context of Feng, Spence, and Pratt (2007) Experiment 2

Context: where transferred from/to

Knowledge

domain

Shooting virtual soldiers vs. locating dots and rotating abstract shapes

Physical

context

Same lab

Temporal

context

Soon after

training

(posttest)

About

5 months late (follow-up)

r

Functional

context

Both clearly research

Social context

Both individual

Modality

Both

computer-based

Based on: Feng, J., Spence, I., & Pratt, J. (2007). Playing an action video game reduces gender differences in spatial cognition. Psychological Science, 18, 850-855.

Similarly, transfer success for these skills may be sensitive to the content of the tasks. For example, would the video game training enhance mental rotation performance measures that are not time-pressured tests? This issue is important, because many aspects of STEM professionals’ work do not require quick responses, but rather deliberate and prolonged thought. Studies such as these represent only the beginning of investigation into these exciting possibilities. Future research might fruitfully investigate transfer situations that involve other aspects of task content and that require far transfer on more of the dimensions highlighted by the simple framework detailed above. Further, many very different sorts of experience fall under the broad label of “video game” (Klopfer, Osterweil, & Salen, 2009) and a wide variety of people play these games, from stereotypical gamers, who dedicate countless hours deeply immersed in their games, to more casual players who play when they happen to be bored, and others who use games as a way to interact with friends. Although the dedicated gamers often come to mind when video games are mentioned, they only represent 11% of players, according to the researchers. Further, multilevel first-person shooter games with lengthy plots and complex graphics, played on a dedicated gaming platform such as an Xbox, offer a very different learning experience from simple driving games, dance-step copying and music-playing games, basic sports simulations (such as Wii tennis), slower moving computer-based simulations (such as managing a family of Sims or building an ancient civilization), socially interactive Internet games played within non-game-specific communities such as Facebook, and cell-phone-based digital versions of board and card games. Intentionally educational variants of these formats would likely offer very diverse learning experiences. These diverse learning experiences also would translate to very different transfer challenges. What all these games have in common is that they have a digital component. As a transfer challenge, some video games might have more in common with chess than with a first-person shooter game, while others might share skills in common with deer hunting. Transfer from different kinds of games needs to be assessed on a case-by-case basis. For all these various forms of learning games, understanding how these learning experiences transfer across the dimensions of content and context detailed earlier should allow us to better evaluate the utility of educational investment in video game learning.

 
Source
< Prev   CONTENTS   Source   Next >