Images versus words
As the carrier of visual information, the computer screen can tempt designers to make excessive use of graphic elements, in particular icons, and especially as buttons for (mostly functional) links. While icons have the advantage of utilising scarce space economically, they can also be difficult to decipher unless they are among the few genuinely established symbols, such as search, print and add to basket. This problem is exacerbated when the user is asked to select one of many icons positioned alongside or below each other as these will be perceived as tiny, barely distinguishable patterns (cf. Boucher 2013: 133).
For these reasons, it may well be advisable to use words rather than images in links. Words convey meaning directly provided they are part of the user’s vocabulary. Since websites are founded on information technology, there is a risk of employing specialist technical terms that mean nothing to many visitors. However, it is possible to build on those terms familiar to any computer user. Where there are valid reasons for an extensive use of icons (e.g., shortage of space), mouseover text boxes can clarify meaning and simplify usage.
One aspect of the user interface design debate among major software suppliers, for whom graphic design contributes to product distinctiveness, also impacts on the web. This is the promotion of a minimalist, “flat” interface for links in reaction to skeuomorphic representations (i.e. simulations of reality) sometimes perceived as awkward . In this way, however, links can easily lose their “affordance”, that is, their ability to make an object’s purpose visible (cf. Boucher 2013: 63-69; see also Section 2.2). Links should be expressed in concise, meaningful language; metalinguistic instructions (e.g., “For xyz click here”) are considered over-complicated (Redish 2012: 259-270). Lack of transparency can also be counteracted by contextualising links externally, by grouping them or adapting them to conventions (e.g., placing them at a specific location). Yet many companies deliberately reject excessively “flat” design, such as Google with its “material design”.
The proliferation of images begs the question of how they should interact with text. Pictures are deployed because they can communicate in ways that are not possible with language; at the same time, words remain a key element because images cannot do everything. Ideally the two will work together, with each type of code contributing its own properties as required. Stockl (2011: 48-49) draws a detailed comparison between the two types of sign. Here are examples from each of the four areas he analyses. First, from a semiotic viewpoint, images are iconic and thus easily perceptible, while language is arbitrary and so harder to perceive. Second, images are perceived simultaneously and holistically, language gradually and in a linear manner. Third, the semantic potential of images is vague and indeterminate, while that of language is (or at least tends to be) precise and definite. Finally, in pragmatic terms, images portray well the relative locations of objects within a space; by contrast, language can explain the logical connections between elements.
In considering the relationship between specific configurations of text and images, and its effect on perception, it is assumed that the two should be melded to form a coherent communicative act. An important indication of how this process might be modelled was provided by Barthes, who was primarily concerned with the semiotics of images themselves. He described that which is depicted as a “message without a code” (1977: 36) and, on another level, identifies connotations deriving from the wealth of cultural experience. As regards the combination with text, he addresses three basic relations: text that specifies the message of the image (“anchorage”); the reverse situation, where an image supports understanding of the text (“illustration”); and a category where both contribute in roughly equal measure to conveying a message (“relay”).
More recently, text-image research has strived to refine its anayltical methodology. Bateman (2014) provides an overview of the state of the art. The list of chapter titles alone indicates the number and the diversity of approaches: “Multimodal relations modelled on account of cohesion”, “[...] modelled on grammar”, “[...] modelled on discourse semantics,” “[...] modelled on accounts of rhetoric” and “[...] based on speech acts, interaction and action” (Bateman 2014:151-163). These approaches also reflect the fact that relations between text and (moving) images are becoming ever more complex and must be embedded in entire text-image networks. In many cases, the relation of a single text to a single image represents just one aspect of such a configuration, or is found in a conventional hypertext module. Text-image analyses work best on the basis of concrete corpora as demonstrated by (amongst others) Stockl (2011), who combines and adapts different approaches in order to analyse the case of advertising. Clearly, it is impossible to make general statements about text-image relations in websites; here, too, closely-defined corpora give the most satisfactory results (see Section 4.1: Runkehl 2011 on banners).