A Multimodal Approach

As explained in the introduction, an SFL approach treats verbal language as the pre-eminent mode of meaning making but recognises that these meanings are also shaped by the context, which may include other semiotic resources integral to the text itself. A multimodal approach takes this one step further by treating these other semiotic resources as complementary semiotic modes (Kress 2010). Multimodal scholars have worked not only on the development of analytical schemas for specific modes, such as visual (Kress and van Leeuwen 2006), film (Bateman and Schidt 2012) and music (Way and McKerrell 2017), but also on the relationships among modes. Hence, in a picture book, the verbal message constitutes one mode or modality while the pictures constitute another. Visual and verbal modes may run in parallel so that visual could be said to illustrate the verbal (or verbal to describe the visual), or there may be a gap between the modes with each contributing slightly different information. This gap between the modes allows for a complementary relationship between them, which can be exploited for such purposes as elaboration or ironic comment (Guijarro 2014; Hayakawa 2014). A gap between modalities can, for example, be used to represent the gap between the self-perceptions of a character in the story and the perception of that character by others. Hence, when Olivia in the picture book Olivia Forms a Band performs as a one-man band for her parents, the line And when she marched in, everyone agreed that Olivia did sound like more than one person’ alone is ambiguous and sounds like an endorsement of her achievement. However, Hayakawa’s illustrations show her family and pets with expressions of horror, and the sound of her instruments is represented by giant words that fill out her side of the page, suggesting a cacophony.

The use of visual and verbal elements will be considered in more detail in the following chapter on manga, but what is relevant here is that visual modalities can be used as evidence that potentially incriminates, or at least undermines, verbal claims. In the case of the televised interview, the multimodal resources are not simply verbal and visual but also include music, voiceovers and video extracts—all of which, I will suggest, contribute to undermining LA’s attempts to redeem himself through his confession. In order to represent this, a description is needed that not only gives consideration to the contributions of these different modes but also provides an account of how they are interrelated. In order to do this, I first describe the initial multimodal montage used to summarise the events relevant to LA’s confession before the interview began, then consider the structural interaction between this, the voiceovers and the interview itself, and finally look at some examples of the interaction between the host and guest—shaped by this immediate multimodal context and created using the resources of television—as a summary of the more general media context in which LA became ensnared. Before doing so, however, it is worth also setting this interview in the context of celebrity confessions.