VII Semiotic and multimodal approaches

Gender and sexuality in discourse Semiotic and multimodal approaches

Michelle M. Lazar (Part VII lead)

Introduction

Multimodality is a field of study in its own right (van Leeuwen 2014). However, it is a field that is ‘rather fragmented and unconsolidated’ (Machin et al. 2016: 302). Taken together, the two statements can be read as attesting positively to the recognition, from various quarters in sociolinguistics and discourse studies, to the fact that meaning-making in texts is a necessarily multiple and heterogeneous semiotic practice. The ‘turn’ towards multimodality (or sometimes referred to as multisemiotics) in linguistics signals an appreciation for the composite nature of texts, in which the meanings of a text could be realised through the integration of more than one semiotic mode (Kress and van Leeuwen 2006). As such, the study of multimodality represents a significant and productive step forward for undertaking a more holistic analysis of texts and talk. It is a move which widens the semiotic lens of linguists to accept that written and spoken language are among several modes of meaning and sensemaking in discourses of any kind. The pervasiveness of digitally-mediated communication nowadays makes the phenomenon of multimodality all the more pronounced and, indeed inevitable, for discourse participants and discourse analysts alike.

Historically, some perspectives within linguistics have been theoretically predisposed to the view of a multiplicity of semiosis than others. Social semiotics is one such perspective (Hodge and Kress 1988; Kress and van Leeuwen 2006; and van Leeuwen 2005) - a body of work which has taken inspiration from the scholarship of Halliday. Halliday’s (1978) book, titled Language as Social Semiotic, paved the way for linguists thinking about language as one, among several, socially embedded semiotic or meaning-making systems; others would include visual symbols and images, gestures, colour, music, materiality, and so on.

The ‘turn’ towards multimodality in language, gender, and sexuality scholarship, more specifically, has been comparatively slow and uneven, although the trend is gradually beginning to change, following shifts in wider sociolinguistics and discourse studies. Still, the relative infancy of multimodal research in language, gender, and sexuality studies seems to suggest that assumptions about the primacy of language in human communication has remained strong. This is perhaps quite understandable, given the core orientation of the field, attested to in the foregrounding of ‘language’ in the name of the field, as well as in the titles of related linguistics journals such as Gender and Language and Language and Sexuality. Widening the lens on semiosis does not eschew or undervalue research about language(s) and language-use pertaining to gender and sexuality, which are important in their own right. However, where texts are multimodal, a focus beyond the semiotics of language alone facilitates a richer and more holistic understanding of the ways gender and sexuality are represented in discourse. Depending on the contexts and communicative practices, several different multisemiotic resources could be deployed, among which language is likely to be one.

In this chapter, five selected ways to carry out multimodal research on gender and sexuality are discussed under the headings ‘Goffman’s approach to gender representations’, ‘Critical social semiotic approach’, ‘Approaches to multimodal digital discourses’, ‘Linguistic/semi- otic landscapes approach', and ‘Multimodal approaches to conceptual metaphor'. The term 'approaches’ is used loosely to refer to a range of frameworks, perspectives, and research foci that has been pertinent to the discursive study of gender and sexuality. In this chapter, in introducing each of the five strands, two studies shall be selected to briefly describe how multimodality is used to address a variety of research questions about gender and sexuality. Among the studies described are the four chapters featured in this section of the book, namely, under the strands on Goffman, critical social semiotics and multimodal digital discourses. Although discussion of the five strands will be dealt with separately, the strands are not mutually exclusive and, in fact, may be combined depending on the nature of one's research projects. On a personal note, even though my own multimodal research falls mainly within critical social semiotics, I have drawn upon the other frameworks and perspectives as well where relevant in different research projects, and have found them to be complementary and mutually beneficial.

Before proceeding to outline the five approaches, the concept of ‘affordance’, which underlies any study of multimodality, needs some explanation. The social semiotic perspective provides a useful understanding of the concept, which may be applied to multimodal analyses more generally. Even though all semiotic modes can be deployed for meaningmaking, they have different ‘affordances’ i.e. each has a different set of potentialities and limitations for meaning-making. Kress and van Leeuwen (2006) explain that semiotic modes are shaped by the intrinsic characteristics and possibilities of the medium as well as by the sociocultural values, histories, and communicative purposes that strongly affect the uses of these potentialities. Not everything that can be expressed in written language, for example, can be necessarily expressed through visual images, or vice versa. Even when seemingly the ‘same’ meanings may be expressed in either pictorial form or in writing or speech, the meanings will be realised differently depending on the medium, which in turn will affect the actual meanings communicated in context. In multimodal analysis, therefore, the economy of meaning-making in texts and talk, contributed by various semiotic modes through their different affordances, becomes pertinent. In what follows, the five approaches to semiotic and multimodal analyses of gender and sexuality shall be presented.

Semiotic and multimodal approaches to the study of gender and sexuality

G off man's approach to gender representations

Goffman’s Gender Advertisements (1979) presents a classic framework for the study of visual representations of gender stereotyping in print media. Although some of the findings are now dated and his methodology is controversial (as will be discussed later), his is a touchstone framework for scholars working on hetero-gendered visual representations, particularly in advertising. Reflecting the period of study in the 1970s, Goffman identified a series of sexist representational patterns in the way white, North American women were depicted vis-a-vis their male counterparts in North American commercial newspaper and magazine advertisements.

Undergirded by an approach that could be described as social-interactional, a key concept of Goffman’s study is ‘gender display’, which refers to ritual-like portrayals of gendered behaviours and appearances, indexical of male and female social actors’ structural relationships and identities with regard to each other. Goflman notes that mediatised images of women (and men) are styled to appear ‘only natural’ (1979: 89) for an audience already accustomed to such routinised gender expressions in actual social situations. Congruent with contemporary social constructionist and performative theories, Goflman emphasises that both curated and actual poses are socially imbibed performances of gender: 'advertisers conventionalise our conventions, stylize what is already a stylization’, the only difference being that mediatised portrayals are ‘hyper-ritualised’ gender displays (1979: 29). In other words, Goflman is referring to gender representations in advertisements as similar to quotidian performances of socially prevalent hegemonic ideologies, except that in advertisements the displays are even more pronounced.

Goffman’s gender displays are premised upon a mutually constimted hierarchical relationship between men and women, patterned after parent-child relationships of benign dominance and subordination, respectively. His analytical framework for interpreting the social dynamics of asymmetrical gender relations relies on simple, often taken-for-granted postures and gestures involving social actors’ use of hands, eyes, facial expressions and head postures, knees, and relative sizing and positioning of bodies. Based on these, he identified six categories of nonverbal gender display in advertisements: (i) relative size (women depicted smaller/lower relative to men); (ii) function ranking (men in executive roles, women in supportive roles); (iii) feminine touch (women’s fingers and hands shown to lightly caress objects or themselves); (iv) ritualisation of subordination (women in submissive and appeasing postures and gestures); (v) licensed withdrawal (women as psychologically removed from situations); and (vi) family scenes (women as mothers).

Goffman’s method of data collection, however, has been criticised; instead of drawing on a random sample, he had selectively chosen data from newspapers and magazines that would match his specific research objectives (Kang 1997). Notwithstanding this, his study has been widely adapted by other scholars. Subsequent studies have improved upon his sampling method, and used his framework to compare his findings with other data sets across time and audience types (e.g. Hiramoto and Teo 2015; Kang 1997), or combined his framework with other (social semiotic) approaches (e.g. Bell and Milic 2002; Lazar 2000). The two studies discussed in this section, Hovland et al. (2005) and Kohrs and Gill (in this volume), belong to the former category of studies.

Selecting a random (instead of a non-random) sample of advertisements from the highest circulating US and Korean women’s magazines in the year 2000, Hovland et al. (2005) used Goffman’s analytical framework to undertake a comparative study of gendered representations of American advertisements over time, between American and Korean cultures, and between differently aged women in the US and Korea. Hovland et. al augmented Goffman’s six analytical categories of gender display by including a few other emergent categories from the studies of other scholars who had also used Goffman’s framework. Notably, Hovland et. al incorporated the categories ‘Body Display’ (state of models’ undress) and ‘Independence/

Self-assurance’ (overall impression of models' sense of autonomy and poise) introduced by Kang (1997), and ‘Full Facial Prominence’ (models gazing directly at the viewer) developed by McLaughlin and Goulet (1999). Their findings revealed that sexist depictions of women in American magazines had decreased significantly over time, although relative to Korean magazines, stereotypical images of women in the American sample was found to be higher. In terms of whether there would be a greater presence of racially diverse models in ads addressed to younger female audiences than an older target group, Hovland et. al found the American magazine ads to be ‘disturbingly ethnocentric' in general. In magazines addressed to younger Korean women, there was a rising trend of using white Caucasian models, signalling an emphasis on a white or Euro-American beauty standard. In both categories then, white women represented normative standards of feminine beauty - a finding that is consonant with wider feminist literature on the mediatisation of feminine beauty ideals (Kilbourne 1999; Wolf 1991).

In using Goffman’s framework, Hovland et. al found that the framework was weighted predominantly towards negative sexist imagery. In contrast, their study revealed the evolution of some new positive analytical categories depicting female independence and confidence which warranted inclusion in the study of more contemporary gender displays in advertising. However, they noted that Goffman’s approach, which focused on visual images alone, rather than on language as well, proved suitable for undertaking cross-cultural comparisons of gender representations. The framework was evidently productive in extending beyond a focus solely on gender to investigating other intersecting social identity categories such as race/ethnicity also. In fact, Hovland et. al recommended that race ought to be investigated more systematically in regard to all of Goffman’s gender display categories in future.

In a similar vein of investigating the extent to which Goffman’s analytical categories and findings still apply to gender advertisements some 40 years since, Kohrs and Gill’s study (in this part) is based on 200 advertisements collected from upmarket US and UK women’s magazines. Using Goffman’s framework as their point of departure, Kohrs and Gill identify six key units of nonverbal behaviour, which they use as their analytical lens: gaze, posture, gesture, touch, facial expression, and proxemics. The authors caution though that, methodologically, one cannot simply derive meaning from the isolated units of body ‘language’; rather, the meanings should be interpreted contextually. When compared to Goffman’s findings, Kohrs and Gill found that, with the exception of ‘feminine touch’, the rest of Goffman’s categories did not apply in the present time, especially with regard to middle- and upper-class Western female audiences. Unlike Goffman’s dataset, Kohrs and Gill focused on advertisements from women’s magazines only, which featured women predominantly (rather than women and men), thus rendering some of Goffman’s analytical categories non-applicable too. Instead, in line with some of the emergent analytical categories found in Hovland et al.’s (2005) study, Kohrs and Gill noted the rise of a bold and confident female figure in their advertisements. Referring to this subjectivity as ‘confident appearing’, it is expressed through a composite of visual signifiers such as an unsmiling direct gaze at the viewer, a confident stance, heads held high, and striding fonvard. Referring to this as a ‘postfeminist’ subjectivity, the authors note the representation as a far cry from the diminutive, passive, deferential gender displays of women decades ago.

In sum, these studies, which draw upon Goffman’s framework, use it as a point of departure to investigate current regimes of media representation. While acknowledging that pockets of overtly patriarchal gender displays remain, newer gender tropes requiring updated analytical categories are noticed. Also, gender stereotyping in advertising, be that traditional or new, continues to be a productive area of study, especially in regard to intersectional analyses of race, class, age, sexuality, and culture.

While Goffman’s framework focused solely on visual representations of gender, applicable primarily to advertising data, the next approach, critical social semiotics, facilitates a multi-semiotic analysis of a wide range of discourse types.

Critical social semiotic approach

In this section, a critical social semiotic approach, which brings together critical discourse studies and a social semiotic approach to multimodal analysis is discussed. In tenns of theoretical orientation, a multimodal social semiotic approach is consonant with a critical discourse perspective. In fact, the major proponents of multimodal social semiotics, Kress and van Leeuwen (2006), are also critical discourse analysts. Although another notable social semiotic framework for visual analysis by O'Toole (1994) exists, Kress and van Leeuwen’s approach is better known and most frequently used in critical discourse studies (see also van Leeuwen 2014). A critical social semiotic approach (a term which, I suggest, calls attention to the social semiotic theoretical foundations applied to critical discourse scholarship) is nowadays known in critical discourse circles also as ‘critical multimodal discourse analysis’ or ‘multimodal critical discourse analysis’. Unlike Goffman’s framework, critical social semiotics or critical multimodality typically involve the analysis of language along with a range of other forms of semiosis, such as visual images, layout, typography, colour, and materiality.

Kress and van Leeuwen’s social semiotic theory of communication offers an explicit and systematic method for analysing the meanings expressed by syntactic relations between the people, places, and things depicted in images. Kress and van Leeuwen based their approach on Halliday’s (1994) notion of ‘metafunctions’, which Halliday had theorised in his systemic functional grammar of language. The metafunctions refer to three basic ways of using language to communicate, which Halliday termed ’ideational' (representing our experience of the world), ‘interpersonal’ (enacting social interactions), and ‘textual’ (organising messages within the text and in relation to the wider context) meanings. Kress and van Leeuwen (2006) adapted the three metafunctions to apply to all semiotic modes, not only language. Accordingly, therefore, any semiotic mode can typically represent objects and their relations in the world as experienced by people; project particular kinds of social relation and truth values (modality) between the producer, the viewer, and the object represented; and make connections within the text, and between the text and its social context. Central to systemic functional theory is the notion of a system or network of semantic choices, i.e. within each metafunction is a set of choices from which producers of texts select different ways to make meaning. The particular choices made in and across semiotic modes in texts and talk are not neutral but can convey ideological meanings.

In language and gender studies, the usefulness of incorporating multisemiotic analysis into a critical discourse perspective was highlighted in an article titled ‘Gender, discourse and semiotics’ (Lazar 2000). In it was proposed

the uncoupling of the two categories “discourse” and “language” in favour of discourse encompassing semiosis of various kinds (including language). A critical analysis of discourse, in other words, involves a commitment to the analysis of various strands of semiosis that configure in the realisation of particular discoursal meanings in texts.

(Lazar 2000:177)

More recently, a call for critical multimodality research on gender, language, and discourse was made by Machin et al. (2016) in a special issue of the journal Gender and Language (vol. 10, no. 3), which featured a collection of articles using the approach.

The studies by Caldas-Coulthard and McLoughlin, in Part VII, draw on a multimodal social semiotics approach in their (feminist) critical discourse analytical (CDA) perspective. Their two chapters indicate some of the types of research pursued within feminist CDA, namely, sexist ideological representations of women as well as ‘newer’ popular postfeminist discourses (see Lazar 2018). Caldas-Coulthard’s chapter deals with gender stereotyping in representations of women criminals in public discourses. Her analysis of photographic images from the Getty image bank and the language and images in the Brazilian press, especially of white Brazilian women, reveal the sexualisation of women criminals, even though the crimes are not sex-related. The intertwining of sexuality and criminality is achieved representationally through a number of semiotic choices like visual modality and ‘demand’ images (represented participants looking directly at the viewer), fragmentation of the female body and sexualised poses, and the colour of clothing construed as indexing danger and sexiness. The emphasis on sexuality in the Brazilian news reports, Caldas-Coulthard observes, is intended to titillate readers, while at the same time to judge and condemn the women criminals even more harshly. More generally, she finds that when women and men are portrayed differently for similar kinds of criminal acts, women offenders are doubly condemned for their criminality and for their sexuality, or simply for the fact that they are women.

In contrast with Caldas-Coulthard’s investigation of overt sexist ideology, McLoughlin’s chapter examines a purportedly progressive feminist discourse promoted in the commercial media which, from a feminist critical discourse perspective, proves ideologically problematic also. McLoughlin’s study shows how articles in a high-fashion magazine (Vogue) and an advertising campaign for a cosmetics brand by a pharmaceutical company (Boots Walgreen Alliance) marketed Chimamanda Ngozi Adichie, a Nigerian author, as a feminist (or, I would suggest, a postfeminist) sensation in the West. By analysing the media discourses over a period of tune, McLoughlin reveals the representational transformation of Adichie from an award-winning author, a popular feminist icon arising from her TED talk on feminism, to a cosmetics brand ambassador. In the process, the focus gradually shifted from Adichie’s writing to her appearance, as evidenced through her verbal represented discourse and thr ough visual semiotic choices of modality and depictions of background settings and colourful eye-catching clothing, which simultaneously indexed trendiness as well as her African heritage. The transformation is not simply a commercial exploitation of Adichie; rather, her ‘brand’ of neoliberal (post) feminism based on notions of individual empowerment coupled with self-professed love for make-up and fashion axe in sync with the media industries’ predisposition towards a non-threatening ‘saleable’ feminism. While the Adichie sensation admittedly contributed towards mainstreaming feminism, her paxticipation in conunodity feminism does a disservice as well. As McLoughlin notes, Adichie’s make-up-wearing stance presented as a feminist act of resistance, ironically, does nothing to address restrictive media ideals of beauty, while painting feminism in rather narrow terms.

With the arrival of new digital media technologies, and their affordances, approaches to multimodal investigations of gender and sexuality on social media have arisen, to which we Uirn our attention next.

Approaches to multimodal digital discourses

Although digital discourse studies is not a new development - known as ‘computer-xnedi- ated discourse analysis’ (Herring 2001) in an earlier period - there has been a surge of interest in this field over the past decade, with the rapid development of digital technologies (collectively referred to as Web 2.0) (e.g. Barton and Lee 2013; Myers 2010; Tannen and Trester 2012; Zappavigna 2012). The advent of newer technologies has brought about innovative possibilities for representation and has spawned an intensely interactive participatory online culture through affordances of generating, networking, and sharing user-generated content. Much of this research has focused on language on social media platforms, although smdies on visual semiotics and multimodality have begun to emerge more recently (Danesi 2017; Jones et al. 2015; Page 2019, for example).

Similarly, in the field of language, gender, and sexuality, studies in digitally mediated contexts have focused primarily on language (e.g. Herring 1995,2003; Mackenzie and Zhao 2020), with a few studies emerging recently on visuality and multimodality of gender and sexuality on social media (e.g. Thurlow 2017; Zappavigna and Zhao 2017). Given the diversity of types of online discourse and the technological and semiotic affordances, there is no singular methodological approach for the analysis of digital discourses; instead, scholars tend towards mixed methodologies (Bolander and Locher 2014). Such is the case, also, for the two studies discussed in Part VII. As the studies show, digitally mediated contexts provide individuals, as well as social groups, with a range of technological and semiotic affordances for performing, sharing, and contesting gender and sexual identities in different and complex ways

Leppanen and Tapionkaski (in this part) examine the identity work of young men in participatory online media culture. Specifically, the authors are interested in how participants mobilise a range of semiotic resources to perform and stylise gender, sexual, and other social identities of the self and others on social media platforms. Adopting a combined discourse analytical and ethnographic approach, Leppanen and Tapionkaski report on the complex ways identities are constructed by different groups of men in two of their previous smdies. In one of these smdies, the performance of a young alternative masculinity termed ‘bronies’ is presented on a discussion forum for young adult male fans of a children’s television series targeted at young girls. Through a combination of language styles, verbal narratives, ernoji, and images, the multifaceted, lived experiences of the 'brony’ fan identity at the intersection of gender, sexuality, and age is enacted, which eschews hegemonic Finnish masculinity. Reporting on another study, the authors discuss the multimodal social media practices of young migrant men in Finland. In this study, self-crafted video narratives by migrant men, shot through smartphones or video cameras, are disseminated on such sites as YouTube and Vimeo. The videos, which are parodic in nature, deploy semiosis such as embodied performances, multilingual language practices, cinematic devices, and music to construct narratives of ‘selves’ and Finnish hostile ‘others’ at the intersection of race, ethnicity, nationality, gender, and sexuality. Leppanen and Tapionkaski’s analyses of intersectional identities enacted through the multisemiotic affordances on digital media transcend binary identity categories such as female-male, feminine-masculine, child-adult, heterosexual- asexual, non-Finnish-Finnish, and humorous-critical, in showing the ways social media participants consciously transgress norms.

While Leppanen and Tapionkaski focus on how individuals using social media platforms construct identities and interact online, Lazar's (2020) study deals with the multimodal digital discourse of an LGBTQ social movement in Singapore. Known as Pink Dot SG (Singapore), the social movement regularly uploads promotional videos featuring Singaporean LGBTQ issues and announcing its annual mass gathering, on the videosharing website YouTube for national and transnational consumption. The study examines the multimodal construction of a homonationalist discourse in the videos. Although in its original sense ‘homonationalism' refers to the instrumental mobilisation of queer identities to serve depoliticised neoliberal agendas of (particularly Western) nation-states, Lazar’s study takes a different spin on the concept, which shows how a disenfranchised social group consciously aligns its queer identity with nationalistic values as a resistive (albeit, assimila- tionist) political strategy. The enmeshing of sexual/gender non-normativity and nationalism in Pink Dot’s videos is achieved multi- and inter-semiotically through the co-deployment of spoken and written language, nationalistic songs, colour symbolism (pink to simultaneously signify national and gay identities), visual images, photographic footages of the national flag, national indexicals (national institutions that index nation-ness), iconic urban cityscapes of Singapore, and embodied performances (or 'corporeal semiotics’) involving the aggregation of citizens to form a human ‘pink dot’.

In the next section, our focus shifts to how linguistic landscape approaches that draw on semiotic and multimodal resources are mobilised for the critical investigation of gender and sexuality concerns.

Approaches to linguistic/semiotic landscapes

Linguistic landscapes has been a burgeoning field of research for more than over a decade, drawing on a wide variety of theoretical and methodological approaches that can be quantitative as well as qualitative in nature. Shohamy and Ben-Rafael (2015: 1) define linguistic landscapes as 'research about the presence, representation, meanings and interpretation of languages displayed in public places’. Landiy and Bourhis’ (1997) work is regarded as the first major effort to refer to publicly displayed signs (e.g. road signs, advertising billboards, street and place names, signs on shops and other buildings) as constitutive of the linguistic landscape of any given place. Over the years, the field has developed in a number of ways. Although the display of different languages in physical environments has been an important focus of linguistic landscape scholarship, the scope has widened beyond a focus on multilingualism, language, and ‘languaging’ to study a range of multimodal representations, artefacts, and materialities in public spaces. In other words, the term ‘linguistic landscapes’ has now been extended to cover multi-semiotic landscapes as well. It could be argued that the terms ‘semiotic landscapes’ (Jaworski and Thurlow 2010) or ‘semioscapes’ (Lazar 2018) might be more appropriate or accurate descriptors to highlight the role played by all forms of semiosis in public spaces, whereby language (as one semiotic system) may or may not play a prominent role as the particular case may be. Whichever is the preferred terminology, linguistic/semiotic landscapes is not only about the study of semiotics in place but how spaces are themselves performed semiotically i.e. spaces accrue particular social meanings through the process of semiosis (Lazar 2018; Milani 2014). Places and spaces, too, have been widened in current linguistic/semiotic landscape studies to encompass the internet as a virtual linguistic landscape (e.g. Biro 2018; Ivkovic and Lotherington 2009) as well as the body as a corporeal landscape (Peck and Stroud 2015).

Contributing to the ongoing development of the field in recent years, scholars have begun to call attention to the gendered and sexualised nature of semioticised public spaces. A special issue on ‘Gender, sexuality and linguistic landscapes' edited by Milani (2018) in the Journal of Linguistic Landscapes is a case in point (see also Kerry 2017; Poseiko 2016). These studies not only contribute to linguistic/semiotic landscape studies, but enrich the field of language, gender, and sexuality by showing how power relations undergirding social orders of gender and sexuality structure, maintain, and/or contest public spaces semiotically. The two studies that will be described demonstrate this well.

In a study about storefront signages (a conventional object of study in linguistic/semi- otic landscape studies), which combine words and symbols, Trinch and Snajdr (2018) focus on how gentrifying women (mostly highly educated, professional, white, and married) straggle to claim and transform the public space in Brooklyn, USA. Drawing on visual ethnography, interviews, and digital archival material, the study looks at the innovative wordplay found on shop signs catered to consumers who are mothers. The authors argue that through the creative signages, Brooklyn mothers attempt to transform traditional notions of motherhood, conservative heterosexual mores, conventional gender roles, as well as the linguistic landscape of the urban space itself. For example, a storefront signage of a local store - 'boing boing: at your cervix since 1996’ - creatively alludes to sexual activity and female reproduction through wordplay as well as filling in the interior of the two ‘o’ graphemes in ‘boing boing’ so that visually the letters resemble cervical caps or diaphragms. By taking to the streets traditionally private and taboo topics, the women are seen as asserting control over their sexuality and mothering. However, as Trinch and Snajdr argue, such attempts were met with public backlash, prompting pragmatic trade-offs in later developments of signages. For instance, the authors note that the ambiguity of wordplay led to the production of a signage with racialised undertones, used as a cover by white women to challenge normative motherhood. The crux of the study is that the signages represented more than a semiotics of creativity; they were, in the words of the authors, ‘formidable social acts’ in a straggle over space and power.

In a different study, Milani (2014) focused on a small data set of ‘banal sexed signs’ or mundane semiotic aggregates, which he had observed in three separate cities. Adopting queer theory as his theoretical lens, Milani discusses the operations of power, in relation to gender and sexuality in unassuming public spaces. Queer theory problematises any form of normativity, including naturalised configurations of sex, gender, and sexuality, and as Milani’s study shows, opens up interpretative ambiguity to some extent. A newsstand at a US airport constituted Milani's first set of data, in which he deconstructed the ideology of gendered difference made manifest through the spatial arrangement of magazines aimed at women and men, and their respective magazine front cover images. Interpellating browsers in distinctly gendered terms, Milani argues that the visual representations of the models on the front pages could potentially offer a 'queer hope of ambiguous desire’. The erotic ambiguity, however, was quickly dispelled through the heteronormative content of the magazines. The second data involved the window display of two T-shirts with affective statements (T love [x]’) printed on them in a retail store in Stockholm. From the outside of the shop, the words and symbols on the two T-shirts conveyed a distinctly heteronormative gender ideology. Yet, when browsing inside the shop, a more dynamic affective process is activated thr ough the availability of T-shirts (albeit less visibly placed) that catered to nonheterosexual desiring persons. Moving from an investigation of material objects in public spaces to store owners own ‘take’ on space, politics, identity, and desire in relation to gender and sexuality, Milani’s final data involved signage in a coffee shop in a Johannesburg suburb as well as interviews with the co-owner of the shop. The signage at the entrance of the coffee shop spelt out a list of ideologically intolerant types of persons disallowed on the premises, aimed at fostering a ‘queer space’ of respectful convivial debate. The queerness of the space was intended by the owners to transcend a reductively ‘gay and lesbian’ space, even though the owners self-identified as lesbians.

The final approach dealt with in this chapter that is useful for gender and sexuality research is multimodal concepmal metaphor analysis.

Multimodal approaches to conceptual metaphor

Unlike popular understandings of metaphor as a literary device using words deliberately to achieve some artistic or rhetorical effect, a cognitive linguistic approach to metaphors, originally developed by Lakoff and Johnson (1980), views metaphors as conceptual in nature, used to facilitate understanding of abstract and unfamiliar ideas in ordinary everyday situations. In fact, conceptual metaphor scholars regard metaphors as a pervasive process in human thought, action, and everyday communication (Kovecses 2002; Lakoff and Johnson 1980). According to conceptual metaphor theory, one conceptual domain is understood in relation to another conceptual domain. For example, in life is a journey, 'life’ is the 'Target' domain in need of comprehension (something abstract and complex) which is understood in terms of a ‘journey’, the ‘Source’ domain (something more familiar) that helps make sense of the former. The process by which metaphoric comprehension is achieved is called ‘mapping’, whereby elements from the Source domain are transferred onto the Target domain, usually systematically and unconsciously. Importantly, conceptual metaphors not only facilitate understanding of a more abstract domain but also mediate and structure the experience of it (Knowles and Moon 2006; Kovecses 2002).

Linguistic, or the verbal, is the most familiar semiotic mode of metaphoric expression. However, as metaphor (following the cognitive approach) is conceptual in nature, it can be expressed nonverbally and multimodally as well (e.g. Forceville 1998; Kovecses 2002). Regardless of whether it is verbal or nonverbal, if only one mode is deployed for metaphoric expression, it is regarded as a monomodal metaphor; whereas in multimodal metaphors the Target and Source domains are predominantly presented in different modes, which can include visuals, written/spoken language, gestures, nonverbal sounds, and music (Forceville 2009).

In language, gender, and sexuality studies, a focus on verbal metaphors in discourse has been more common (e.g. Hobbs 2013; Roller 2004; Velasco-Sacristan and Fuertes-Olivera 2006) than multimodal metaphors. The latter, in fact, remains as yet an untapped research approach, which can be better utilised for investigating topics on discourse, gender, and sexuality. The two studies discussed in this section represent one of the earliest in multimodal metaphor analysis involving gender and advertising (Lazar 2009) and a more recent one on sex education material for preschool children (Liang et al. 2017). The studies will be discussed in reverse order.

Drawing on Forceville’s study ofpictorial and multimodal metaphors, Liang, O’Halloran, and Tan (2016) examine metaphorical representations in sex education picture books aimed at preschoolers in mainland China. The use of metaphors in this educational discourse genre ostensibly served the pedagogical objective of facilitating better understanding of scientific knowledge about sex and sexuality in a young audience. Yet, the authors argue that through the metaphorical representations deployed in the books, ideologies about gender stereotypes and traditional mainstream values on (heteronormative) sex and sexuality were being instilled in the young at the same time. One type of metaphor based on personification grafted male or female faces and traits onto the target domain concepts of reproductive organs, sperm cells, and ova, so that binary sex categories and gender stereotyping became naturalised. Another type, identified as domestication metaphors, facilitated understanding of distant and abstract biological concepts and processes by associating them with familiar and concrete objects within the worldview of children. However, the selection of source domains was sometimes ideologically laden (for example, the ovum was verbally and visually metaphorised as a treasure quest). Finally, the authors identified ‘cross-experience’ metaphors, which were established multimodally through what the authors describe as mind-body associations. In this way, conjugal love and procreation were foregrounded and prioritised as the driving force for the physical activity of sexual intercourse.

Set against a general observation that a ‘war mentality’ was becoming a prevalent com- monsensical mode of rationality, thr ough the utilisation of the domain of war to conceptualise a wide range of non-militarised activities, Lazar’s (2009) study focused on how the domain of banal feminine beauty practices has been metaphorised in terms of warfare in advertising discourse. Combining a critical discourse perspective with multimodal metaphor analysis, the study based on over 100 print beauty advertisements (ads for cosmetics, skin and haircare products, slimming and body modification services), uncovered the operation of the conceptual metaphor beautification is war, expressed through a combination of language, images, and colour. In tandem with a problem-solution schema commonly found in advertising, a multi-tiered analysis of the conceptual metaphor was systematically presented. ‘Problems’ were anything that hindered the achievement of the beauty ideal (e.g. nature or ageing), which were conceptualised as enemies. ‘Solutions' were cosmetic brands that occupied the role of powerful allies to prospective consumers. The ‘consumers' were women conceptualised both as fighters in the struggle and whose bodies, at the same time, were the battlegrounds. The study revealed contradictory elements in the construction of ‘modem’ femininity. While on the one hand, women were represented as ‘empowered’ subjects, on the other hand, militarisation associated women’s exercise of agency with a hegemonic mode of masculinity. With women’s own bodies cast as sites of straggle, and a concomitantly radical shift in conceptualisation from seeing the ‘enemy-as-other’ to the ‘enemy-as-self’, the study suggested the possibility that anxiety and alienation in women’s relationships with their own bodies as threats was exigent.

Conclusion

In this chapter, five approaches or lenses for the study of semiotics and multimodality were presented. These are by no means comprehensive; rather, they were selected to highlight productive ways for analysing semiotic representations and performances of gender and sexuality in a range of contexts. As the studies discussed in this section showed, perpetuation as well as contestation and subversion of gender/sexual norms and identities were enabled through the mobilisation of a number of semiotic resources (with language playing a salient or not so salient part, depending on the situations). The range of semiosis involved included gestures, colour, visual images and photography, music and song, dress and props, materialities, represented affect, typography, and embodied performances. Each contributed to the meaning-making in texts and talk, in an integrated way, based on their affordances.

Although the turn towards semiotics and multimodality in language, gender, and sexuality scholarship, on the whole, is at the stage of relative infancy, it is slowly but surely growing. Fragmentation and unconsolidation of approaches, therefore, can be viewed positively as a healthy sign of growth and maturation of any field. Messiness and mixing of approaches, I suggest, disavow foreclosing the development of diverse ideas and insights, and offer opportunities for learning, borrowing, and integrating from different perspectives. Multimodality itself, as a field, is a product of (multi-disciplinary) learning from such fields as linguistics, semiology, proxemics, art history, cultural studies, and psychology. If we extend the concept of affordances beyond semiotic modalities to semiotic and multimodal approaches, then we can say that each framework or lens has its own (theoretical and practical) affordances for apprehending the phenomena of multi-semioticity. That, I believe, can only enhance the robustness and diversity of inquiries about gender, sexuality, and other identities in texts and talk.

References

Barton, D. and Lee, C. (2013) Language Online: Investigating Digital Texts and Practices. London: Routledge.

Bell, P. and Milic, M. (2002) ‘Goffman’s gender advertisements revisited: Combining content analysis with semiotic analysis’. Visual Communication, 1(2), pp. 203-222.

Biro, E. (2018) ‘More than a facebook share: Exploring virtual linguistic landscape’. Acta Universitatis Sapientine, Philologica, 10(2), pp. 181-192.

Bolander, B. and Locher, M. A. (2014) ‘Doing sociolinguistic research on computer-mediated data: A review of four methodological issues’. Discourse, Context and Media, 3, pp. 14-26.

Danesi, M. (2017) The Semiotics of Emoji: The Rise of Visual Language in the Age of the Internet. London: Bloomsbury.

Goffman, E. (1979) Gender Advertisements. Cambridge, MA: Harvard University Press.

Forceville, C. (1998) Pictorial Metaphor in Advertising. London: Routledge.

Forceville, C. (2009) ‘Non-verbal and multimodal metaphor in a cognitivist framework agendas for research’. In: Forceville, C. and Urios-Aparisi, E. (eds.) Multimodal Metaphor. Berlin: Mouton de Gruyter, pp. 18-42.

Halliday, M. A. K. (1978) Language as Social Semiotic. London: Arnold.

Halliday, M. A. K. (1994) ‘Systemic Theory’, in Asher, R. E. (ed) Encyclopedia of Language and Linguistics, vol 8. Pergamon Press.

Herring, S. (1995) ‘Men’s language on the internet’. Nordlyd, 23, pp. 1-20.

Herring, S. (2001) ‘Computer-mediated discourse’. In: Schiffrin, D., Tannen, D., and Hamilton, H. (eds.) The Handbook of Discourse Analysis. Oxford: Blackwell, pp. 612-634.

Herring, S. (2003) ‘Gender and power in on-line communication’. In: Hohnes, J. and Meyerhoff, M. (eds.) Handbook of Language and Gender. Malden, Oxford: Blackwell, pp. 202-228.

Hiramoto, M. and Teo, C. S. L. (2015) ‘Heteronormative love makes a house a home: Multimodal analysis of luxury housing advertisements in Singapore’. Journal of Language and Sexuality, 4(2), pp. 223-253.

Hobbs, P. (2013) ‘Fuck as a metaphor for male sexual aggression’. Gender and Language, 7(2), pp. 147-174.

Hodge, R. and Kress, G. (1988). Social Semiotics. Cambridge: Polity.

Hovland, R., McMahan, C., Lee, G., Hwang, J. S., and Kim, J. (2005) ‘Gender role portrayals in American and Korean advertisements’. Sex Roles, 53(11/12), pp. 887-899.

Irkovic, D. and Lotherington, H. (2009) ‘Multilingualism in cyberspace: Conceptualizing the virtual linguistic landscape’. Journal of Multilingualism, 6(1), pp. 17-36.

Jaworski, A. and Thurlow, C. (2010) ‘Introducing semiotic landscapes’. In: Jaworski, A. and Thurlow, C. (eds.) Semiotic Landscapes: Language, Image, Space. London: Continuum, pp. 1-40.

Jones, R., Chik, A., and Hafner, C. (eds.) (2015) Discourse and Digital Practices: Doing Discourse Analysis in the Digital Age. London: Routledge.

Kang, M. E. (1997) ‘The portrayal of women’s images in magazine adverts: Goffinan’s gender analysis revisited’. Sex Roles, 37(11/12), pp. 979-993.

Kerry, V. (2017) ‘The construction of hegemonic masculinity in the semiotic landscape of a CrossFit “Cave”’. Visual Communication, 16(2), pp. 209-237.

Kilbourne, J. (1999) Cant Buy My Love: How Advertising Changes the Way We Think and Feel. New York, NY: Simon & Schuster.

Knowles, M. and Moon, R. (2006) Introducing Metaphor. London: Routledge.

Koller, V. (2004) Metaphor and Gender in Business Media Discourse: A Critical Cognitive Study. Basingstoke: Palgrave Macmillan.

Kovecses, Z. (2002) Metaphor: A Practical Introduction. Oxford: Oxford University Press.

Kress, G. and van Leeuwen, T. (2006) Reading Images: The Grammar of Visual Design, 2nd ed. London: Routledge.

Lakoff, G. and Johnson, M. (1980) Metaphors We Live By. Chicago: University of Chicago Press.

Landry, R. and Bourhis, R. Y. (1997) ‘Linguistic Landscape and Ethnolinguistic Vitality: An Empirical Study’, Journal of Language and Social Psychology’, 16(1), pp. 23-49.

Lazar, M. M. (2000) ‘Gender, discourse and semiotics: The politics of parenthood representations’. Discourse and Society, 11(3), pp. 373-400.

Lazar, M. M. (2009) ‘Gender, war and body politics: A critical multimodal analysis of metaphor in advertising’. In: Ahrens, K. (ed.) Politics, Gender and Conceptual Metaphors. Basingstoke: Palgrave, pp. 209-234.

Lazar, M. M. (2018) ‘Gender, sexuality and semioscapes’. Linguistic Landscape, 4(3), pp. 320-327.

Lazar, M. M. (2020) ‘Semiotics of homonationalism’. In: Hall, K. and Barrett, R. (eds.) Oxford Handbook of Language and Sexuality. Oxford: Oxford University Press.

Liang, J. Y., O’Halloran, K., and Tan, S. (2016) ‘Where do I come from? Metaphors in sex education picture books for young children in China’. Metaphor and Symbol, 31(3), pp. 179-193.

Machin, D., Caldas-Coulthard, C., and Milani, T. (2016) ‘Doing critical multimodality in research on gender, language and discourse’. Gender and Language, 10(3), pp. 301-308.

Mackenzie, J. and Zhao, S. (eds) (2020) ‘Special issue - Doing motherhood online: Parenting, identity and digital interaction’, Discourse, Context, & Media, 37/38.

McLaughlin, T. L. and Goulet, N. (1999) ‘Gender Advertisements in Magazines Aimed at African Americans: A Comparison to Their Occurrence in Magazines Aimed at Caucasians’, Sex Roles, 40, pp. 61-71.

Milani, T. (2014) ‘Sexed signs - Queering the scenery’. International Journal of the Sociology’ of Language, 228, pp. 201-225.

Milani, T. (ed) (2018) Gender, Sexuality and Linguistic Landscapes. Special Issue: Journal of Linguistic Landscapes. Amsterdam: John Benjamins.

Myers, G. (2010) The Discourse of Blogs and Wikis. London: Continuum.

O’Toole, M. (1994) The Language of Displayed Art. London: Leicester University Press.

Page, R. (2019) ‘Group selfies and snapchat: From sociality to synthetic collectivisation’. Discourse, Context and Media, 28, pp. 79-92.

Peck, A. and Stroud, C. (2015) ‘Skinscapes’. Linguistic Landscape, 1(1/2), pp. 133-151.

Poseiko, S. (2016) ‘Images of women in the semiotic landscape of the Baltic states’. Miscellanea Anthropologica et Sociologica, 17(3), pp. 111-128.

Shohamy, E. and Ben-Rafael, E. (2015) ‘Introduction - Linguistic landscape: A new journal’. Linguistic Landscape, 1(1/2), pp. 1-5.

Tannen, D. and Trester, A. M. (eds.) (2012) Discourse 2.0: Language and New Media. Washington, DC: Georgetown University Press.

Thurlow, C. (2017) ‘Forget about the words? Tacking with language, media and semiotic ideologies of digital discourse: Case of sexting’. Discourse, Context and Media, 20, pp. 10-19.

Trinch, S. and Snajdr, E. (2018) ‘Mothering Brooklyn: Signs, sexuality, and gentrification under cover’. Linguistic Landscape, 4(3), pp. 214-237.

Van Leeuwen, T. (2014) ‘Critical discourse analysis and multimodality’. In: Hart, C. and Cap, P. (eds.) Contemporaiy Critical Discourse Studies. London: Bloomsbury, pp. 281-297.

Van Leeuwen, T. (2005) Introducing Social Semiotics. London: Routledge.

Velasco-Sacristan, M. and Fuertes-Olivera, P. (2006) ‘Towards a critical cognitive-pragmatic approach to gender metaphors in advertising English’. Journal of Pragmatics, 38(11), pp. 1982-2002.

Wolf, N. (1991) The Beauty’ Myth: How Images of Beauty’ are Used against Women. New York, NY: Anchor Books.

Zappavigna, M. (2012) Twitter and Social Media: How We Use Language to Create Affiliation on the Web. London: Continuum.

Zappavigna, M. and Zhao, S. (2017) ‘Selfies on “mommyblogging”: An emerging visual genre’. Discourse, Context and Media, 20, pp. 239-247.