Loading...
Title: Metaphoricity of Conventionalized Diegetic Images in Comics

Metaphoricity of Conventionalized Diegetic Images in Comics

A Study in Multimodal Cognitive Linguistics

by Michał Szawerna (Author)
©2017 Monographs 487 Pages
Linguistics
Series: Łódź Studies in Language, Volume 54

Summary

This book offers a cognitive-semiotic approach to metaphoricity of visual representations in static visual narratives referred to as comics. It implements this approach in an exploration of conventionalized visual signs depicting diegetic situations, motion events, sound events, and diverse psychological experiences in such narratives. With his focus on the intersection of comics studies, conceptual metaphor theory, and Charles Sanders Peirce’s theory of signs, the author analyzes a broad array of attested data retrieved from comics exemplifying various publication formats, generic conventions, and cultural traditions. His exploration situates the metaphoricity of the analyzed visual signs against the backdrop of their overall semiotic makeup and in relation to the metaphoricity of their linguistic counterparts.

Table Of Contents

  • Cover
  • Title
  • Copyright
  • About the author(s)/editor(s)
  • About the book
  • This eBook can be cited
  • Table of contents
  • Introduction
  • Chapter 1. Theoretical orientation
  • 1.0. Introduction
  • 1.1. An overview of the current research into the meaning-making potential of comics
  • 1.1.1. The origins and early stages of comics scholarship
  • 1.1.2. The new comics scholarship
  • 1.1.2.1. Seminal contributions from practicing cartoonists
  • 1.1.2.2. Internationalization of the new comics scholarship
  • 1.1.2.3. Popularization of research into comics
  • 1.1.2.4. The formalist strand of the new comics scholarship
  • 1.2. An outline of Peirce’s sign theory
  • 1.2.1. The correlates of the sign and continuous semiosis
  • 1.2.2. Classification of signs
  • 1.2.2.1. The first trichotomy: qualisigns, sinsigns, and legisigns
  • 1.2.2.2. The second trichotomy: icons, indexes, and symbols
  • 1.2.2.3. The third trichotomy: rhemes, dicisigns, and arguments
  • 1.3. An outline of conceptual metaphor theory
  • 1.3.1. The origins of conceptual metaphor theory
  • 1.3.2. Main elements of conceptual metaphor theory
  • 1.3.2.1. The principle of unidirectionality
  • 1.3.2.2. The dichotomy between source and target domains
  • 1.3.2.3. Orientational, ontological, and structural metaphors
  • 1.3.2.4. Conceptual metaphors and image schemas
  • 1.3.2.5. Image-schematic metaphors and other kinds of conceptual metaphor as overlapping categories
  • 1.3.2.6. Other characteristics of conceptual metaphors
  • 1.3.2.7. The invariance hypothesis/principle and its limitations
  • 1.3.3. Primary metaphor theory
  • 1.3.4. Objectification theory
  • 1.3.5. Interaction of metaphor and metonymy
  • 1.3.6. Linguistic and nonlinguistic manifestations of conceptual metaphors
  • 1.4. On the compatibility of Peirce’s conception of metaphor with conceptual metaphor theory
  • 1.5. An outline of a cognitive-semiotic approach to the study of metaphorical signs in comics
  • Chapter 2. Metaphoricity of individual comics panels and multi-panel complexes
  • 2.0. Introduction
  • 2.1. A general characterization of panels and multiframes
  • 2.1.1. Ontology of the panel
  • 2.1.2. Formal parameters of panels
  • 2.1.3. Panels as units of increasingly inclusive formal structures
  • 2.1.4. Spatial and temporal interrelatedness of panels
  • 2.1.5. Fragmentariness of multi-panel structures and their interpretation
  • 2.2. A Peircean semiotic characterization of panels and multiframes
  • 2.2.1. Imaginal and diagrammatic hypoiconicity of panels
  • 2.2.2. Indexicality of panels and multi-panel complexes
  • 2.2.3. Symbolicity of panels and multi-panel complexes
  • 2.2.4. Metaphorical hypoiconicity of panels and multi-panel complexes
  • 2.3. Metaphors characterizing individual panels and their verbal manifestations
  • 2.3.1. Metaphorical reifications of episodic situations in comics and language
  • 2.3.1.1. Metaphorical reification of episodic situations as objects in comics and language
  • 2.3.1.2. Metaphorical reification of episodic situations as containers in comics and language
  • 2.3.2. Metaphorization of visual fields as containers in comics and language
  • 2.3.3. Metaphorization of the means of accessing information as windows in comics and language
  • 2.4. Metaphors characterizing multi-panel structures and their verbal manifestations
  • 2.5. Discussion
  • Chapter 3. Metaphoricity of conventional representations of diegetic motion in comics
  • 3.0. Introduction
  • 3.1. Principal varieties of visual representations of diegetic motion used in comics
  • 3.2. Representational conventions included in the scope of this chapter and terminological matters
  • 3.3. A semiotic characterization of motion-cueing images of bodily configurations
  • 3.3.1. The semiotic makeup of pictorial representations of characters in comics
  • 3.3.2. The semiotic makeup of motion-cueing images of gestures
  • 3.3.3. The semiotic makeup of motion-cueing images of whole body actions
  • 3.3.4. The semiotic makeup of motion-cueing images of facial expressions
  • 3.4. A conceptualist characterization of motion-cueing images of bodily configurations
  • 3.5. Ontology of polymorphic motion signs
  • 3.6. A semiotic characterization of polymorphic motion signs
  • 3.7. Polymorphic motion signs as metaphorical reifications of episodic motion events
  • 3.8. Metaphor-metonymy interaction in polymorphic motion signs
  • 3.9. Ontology of polyptychal motion signs
  • 3.10. A semiotic characterization of polyptychal motion signs
  • 3.11. Polyptychal motion signs as metaphorical reifications of episodic motion events
  • 3.12. Metaphor-metonymy interaction in polyptychal motion signs
  • 3.13. Ontology of motion signs comprising an image of the moving body and one or more motion lines
  • 3.14. A semiotic characterization of motion signs comprising an image of the moving body and motion lines
  • 3.15. Motion signs comprising an image of the moving body and motion lines as image-schematic reifications of episodic motion events
  • 3.16. Discussion
  • Chapter 4. Metaphoricity of conventional representations of diegetic sound in comics
  • 4.0. Introduction
  • 4.1. An overview of the major kinds of visual signs representing diegetic sound in comics
  • 4.1.1. Structural heterogeneity of visual signs representing diegetic sound in comics
  • 4.1.2. Expressive potential of visual signs representing diegetic sound in comics
  • 4.1.3. Contextual situatedness of visual signs representing diegetic sound in comics
  • 4.2. Representational conventions included in the analytical scope of this chapter
  • 4.3. Conventional representations of speech sounds produced by characters in diegetic worlds of comics
  • 4.3.1. Ontology of speech balloons
  • 4.3.2. A semiotic characterization of the speech balloon conceived of as a sound sign
  • 4.3.3. A semiotic characterization of the components of the speech balloon conceived of as sound signs
  • 4.3.3.1. Balloon-internal writing as a visual representation of the acoustic/auditory form of speech
  • 4.3.3.2. Non-standard typography of balloon-internal writing as a representation of the acoustic/auditory form of speech
  • 4.3.3.3. Balloon-internal pictograms as representations of the acoustic/auditory form of speech
  • 4.3.3.4. Interaction of the linguistic and non-linguistic contents of speech balloons conceived of as sound signs
  • 4.3.3.5. The shape of the speech balloon as a representation of the acoustic/auditory form of speech
  • 4.3.3.6. The color of the speech balloon as a representation of the acoustic/auditory form of speech
  • 4.3.4. A semiotic characterization of speech balloon complexes conceived of as sound signs
  • 4.3.5. A semiotic characterization of stand-alone writing conceived of as a representation of the acoustic/auditory form of speech
  • 4.4. Conventional representations of non-speech sounds produced by characters in diegetic worlds of comics
  • 4.4.1. A semiotic characterization of written representations of non-speech vocalizations
  • 4.4.2. A semiotic characterization of stand-alone punctuation marks representing non-speech vocalizations
  • 4.4.3. A semiotic characterization of visual representations of non-speech vocalizations featuring elements of musical notation
  • 4.5. Conventional representations of sounds belonging to the sonic environment of diegetic worlds of comics
  • 4.5.1. A semiotic characterization of written representations of sounds belonging to the sonic environment
  • 4.5.2. A semiotic characterization of (para)balloonic representations of sounds belonging to the sonic environment
  • 4.6. Metaphors characterizing conventional representations of diegetic sound in comics and their verbal exemplifications
  • 4.6.1. Balloons as metaphorical reifications of diegetic sound events
  • 4.6.2. Metaphoricity of the outlines of balloons representing diegetic sound events
  • 4.6.3. Written texts as metaphorical reifications of the audible form of diegetic sound events
  • 4.6.4. Metaphoricity of non-standard typography in written representations of diegetic sound events
  • 4.6.5. Metaphoricity of multi-balloonic representations of the audible form of sequences of diegetic utterances
  • 4.7. Discussion
  • Chapter 5. Metaphoricity of conventional representations of diegetic mental experience in comics
  • 5.0. Introduction
  • 5.1. An overview of prior research on conventional representations of diegetic mental experience in comics
  • 5.1.1. Representations of the meaning of externalized diegetic utterances
  • 5.1.2. Representations of diegetic thought
  • 5.1.3. Representations of diegetic emotions
  • 5.2. Representational conventions included in the scope of analysis
  • 5.3. A semiotic characterization of balloonic representations of the linguistic meaning of diegetic utterances
  • 5.3.1. Written text as a representation of the linguistic meaning of diegetic utterances
  • 5.3.2. Balloon-internal pictograms and pictorial runes as representations of the linguistic meaning of diegetic utterances
  • 5.3.3. Speech balloons conceived of as indexical representations of the sources and goals of the meanings communicated by diegetic speech events
  • 5.4. Speech balloons as visual exemplifications of the metaphor whereby linguistic communication is construed as physical transfer
  • 5.5. A semiotic characterization of visual representations of direct thought and their components
  • 5.6. Metaphorical underpinnings of thought balloons
  • 5.6.1. Thought balloons as metaphorical reifications of inner diegetic utterances
  • 5.6.2. Metaphoricity of the outline of thought balloons representing inner diegetic utterances
  • 5.6.3. Written texts as metaphorical reifications of the non-externalized form of inner diegetic utterances
  • 5.6.4. Metaphoricity of non-standard typography in written representations of inner speech
  • 5.6.5. Metaphoricity of multi-balloonic representations of the non-externalized form of sequences of inner diegetic utterances
  • 5.7. A cognitive-semiotic characterization of thought balloons conceived of as representations of the semantic poles of inner diegetic utterances
  • 5.8. Representations of emotions in the narrative medium of comics
  • 5.8.1. A conceptualist characterization of linguistic representations of emotions
  • 5.8.2. A cognitive-semiotic characterization of representations of emotions in comics
  • 5.8.2.1. Mimetic visual images depicting physiological and behavioral responses to the signified emotions
  • 5.8.2.2. Non-mimetic visual images depicting expressive responses to the signified emotions
  • 5.8.2.3. Non-mimetic visual images depicting metaphorical responses to the signified emotions
  • 5.8.2.4. Non-mimetic visual images depicting emotions via metaphor
  • 5.9. Discussion
  • Conclusion
  • References
  • Index
  • Series index

| 13 →

Introduction

Comics scholars like Christopher Eklund (2006: 209), Randy Duncan and Matthew J. Smith (2009: vii), and Kent Worcester (2010: 111) concur in the opinion that extensive research on static visual narratives referred to collectively as comics has been developing for several decades, but both Worcester (2010: 111) himself and Jeet Heer and Worcester (2009: xi) point out that the heyday of modern comics scholarship began, roughly speaking, at the turn of the 1980s and 1990s, with the emergence of what Heer and Worcester refer to as the “new comics scholarship” (2009: xiv). According to Heer and Worcester (2009: xiii), the new comics scholarship marks an important change in contemporary comics studies — a change toward research into “the formal aspects of comics” (Heer and Worcester 2009: xiv), with the focus on revealing the unique capacity of comics exemplifying various publication formats, generic conventions, and cultural traditions to “achieve meaning” (Heer and Worcester 2009: xiii; original emphasis). The lively dialogue about the capacity of comics for meaning, referred to by Heer and Worcester (2009: xiv) as the “formalist turn” in the new comics scholarship, was initiated by practicing cartoonists — Will Eisner (2008 /1985/), Robert C. Harvey (1994, 1996), and Scott McCloud (1994 /1993/) — whose contribution to comics studies, while invaluable insofar as it provided “a solid foundation for academic inquiry into comics” (Varnum and Gibbons 2001: xiii), was criticized for its lack of an academic orientation, for “being removed from the scholarly traditions with which it might best intersect” (Beaty and Nguyen 2007: vii).

With a view to systematizing their research efforts, various proponents of the formalist turn framed their accounts of the expressive potential of comics in terms of established theories of meaning, and in a number of cases they singled out Peircean semiotics and conceptual metaphor theory as their analytical frameworks.

Given the sheer scope and intricacy of the theory of signs continually formulated and reformulated by Charles Sanders Peirce throughout his intellectual life, the emphasis placed by this theory on “the importance of interpretation to signification” (Atkin 2010 /2006/), and its lack of the logocentric bias that has informed semiotic analyses conducted by neo-Saussurians for a good part of the twentieth century (Kwiatkowska 2011: 314–316; Jappy 2013: x–xi), the choice of Peircean semiotics as the principal framework in which to explore the meaning-making potential of visual signs found in comics seems quite natural. In turn, the application of conceptual metaphor theory to the analysis of the form of comics is motivated on the grounds that this theory offers a principled explanation of the reasons why many non-mimetic signs belonging to the standard expressive repertoire of comics — signs representing emotions, thoughts, sounds, movements, etc., in static visual form — are “intuitively interpreted on first encounter” (Miodrag 2013: 196) despite their highly conventionalized, unrealistic form. More specifically, given the central tenet of conceptual metaphor theory whereby metaphor is “a general cognitive process that may materialize in different modalities” (Müller 2008: 26), it is justified ← 13 | 14 → to hypothesize, as I suggested elsewhere, that the effortlessness with which these non-mimetic signs are interpreted stems, at least in part, from their metaphoricity, that is, “from the fact that they instantiate conceptual metaphors that are familiar, albeit at an unconscious level, to creators as well as readers of comics” (Szawerna 2014: 92). Put differently, a characterization of non-mimetic visual signs found in comics in terms of conceptual metaphor theory goes at least some way toward accounting for the form of these signs, which is conceived of as motivated by the way in which their meaning is captured in accordance with a range of entrenched conceptual metaphors which have a basis in perceptual, social, and cultural experience.

With regard to the first of these established theories of meaning, a number of comics scholars — notably, Anne Magnussen (2000), Catherine Khordoc (2001), Mario Saraceni (2003), Charles Forceville (2005), Duncan and Smith (2009), and Neil Cohn (2010b) — have applied the concepts of Peircean semiotics in their attempts to characterize the meaning-making potential of visual signs found in comics, but it was already McCloud (1994 /1993/: 26–28) who repeatedly used such terms as icon, iconic, and symbol in his discussion of the way visual signs in comics become meaningful. In an earlier publication (Szawerna 2013a: 51–52), I specified the reasons why these Peircean explorations seem less than satisfactory; at this juncture, it will be instructive to report those reasons and briefly expand on them.

As to McCloud (1994 /1993/), he characteristically departs from the Peircean understanding of icon and symbol by redefining these terms in accordance with the way they are intuitively understood by non-experts. Specifically, McCloud defines an icon as “any image used to represent a person, place, thing or idea” (1994 /1993/: 27; original emphasis), and he defines symbols as “images we use to represent concepts, ideas and philosophies” (1994 /1993/: 27; original emphasis). What is more, McCloud explains that, understood in this way, symbols “are one category of icon” (1994 /1993/: 27; original emphasis). Additionally, McCloud (1994 /1993/) altogether overlooks the indexicality of visual signs found in comics, but so do Saraceni (2003) and Cohn (2010b). McCloud (1994 /1993/) does not use the terms index, indexical, or indexicality at all, whereas Saraceni (2003) and Cohn (2010b) do use them, albeit sparingly, but not with reference to the expressive repertoire of comics. On the whole, Saraceni (2003: 20–27) does not expand the Peircean portion of his account beyond the rather obvious iconic similarity between pictorial images found in comics and their real-world referents and the equally obvious symbolicity of writing, whereby the forms of written signs are arbitrarily related to their meanings, and Cohn (2010b: 188–189) limits his Peircean exploration of the visual signs used in the Japanese manga to the observation that they simultaneously exhibit iconic and symbolic characteristics. On the few occasions when indexicality is in fact mentioned, the proposed semiotic analyses of visual signs found in comics still seem less than comprehensive for the reason that the analysts (Duncan and Smith 2009: 10–11; Forceville 2005: 73; Khordoc 2001; Magnussen 2000) rarely characterize ← 14 | 15 → these visual signs at the level subordinate to indexes and icons,1 do not recognize the fact that in comics the semiotic value of a visual sign emerges with relation to multiple referents, and overlook the cyclicity of many visual signs found in comics, whereby a sign gives rise to another sign, which in turn gives rise to yet another sign, potentially ad infinitum.

This fragmentariness causes certain contradictions in the proposed analyses and gives rise to a degree of descriptive vagueness. Regarding the contradictions, Forceville (2005: 73), for example, argues quite categorically that “[s]ince anger is an abstract concept, it by definition defies iconic representation, and can hence only be rendered by means of indexical and symbolic signs.” Later on, however, he characterizes a number of visual signs of anger found in René Goscinny and Albert Uderzo’s Asterix and the Roman Agent (1972 [1970]) as metaphorical representations (Forceville 2005: 80–86), which automatically confers onto them the status of iconic signs — according to Peirce (CP § 2.277), metaphors, along with images and diagrams, belong to the category of iconic signs, also known as hypoicons. Regarding the vagueness, it becomes evident, for example, in the fact that Forceville (2005: 77) treats the visual sign of anger referred to as “smoke,” which comprises multiple pictorial images of smoke puffs placed above the pictorial image of an angry character’s head, on a par with the visual sign of anger referred to as “bold face,” which in turn comprises large sized, emboldened letters making up a written representation of an angry character’s loud speech, as the so-called “pictorial runes,”2 that is, visual signs which are at once indexical and metaphorical (Forceville 2005: 73–74) — without making it clear that they are aptly characterized as converses of each other. The converse relationship between the two pictorial runes consists in the fact that Forceville’s (2005) smoke qualifies as a realistic representation of an unrealistic index, insofar as this visual sign realistically depicts puffs of smoke in one semiotic cycle, and in another the puffs are interpreted as the unrealistic effect of anger conceived of as fire (by virtue of the ANGER IS FIRE metaphor, invoked by Forceville 2005: 82), whereas Forceville’s (2005) bold face is characterizable as an unrealistic representation of a realistic index, insofar as in one semiotic cycle it is an unrealistic, metaphorical sign of loud speech (by virtue of the MORE OF FORM IS MORE OF CONTENT metaphor, invoked by Forceville 2005: 82), and in another the loud speech is recognized as a realistic symptom of anger known from everyday life.3

Despite its shortcomings, however, Forceville’s (2005) exploration of the visual representations of anger in Asterix and the Roman Agent (1972 [1970]) has an ← 15 | 16 → important advantage over other accounts of comics informed by Peirce’s sign theory: it attempts to integrate a Peircean characterization of the analyzed visual signs with the insight into linguistic representations of anger gained from the research done within conceptual metaphor theory. Forceville’s (2005) adoption of this kind of integrated approach has enabled him to develop a relatively broad-based characterization of the metaphorical iconicity of the visual signs of anger encountered in the analyzed comic and avoid the kinds of problematic formulations that are sometimes found in comparable accounts which are framed solely in terms of Peircean semiotics. One such formulation comes from Khordoc’s (2001) article on the visual representations of sound deployed by Goscinny and Uderzo in various installments of the Asterix (1959–1979) series. In her article, Khordoc (2001: 165) characterizes the visual convention whereby “the bigger the letters, the louder the voice, and the smaller the letters, the lower the voice” as “an arbitrary method of indicating directly that the volume is changing.” According to Khordoc, “[t]here is no reason why bigger letters would mean that a voice is louder though it has certainly become convention in comics” (2001: 165). Pace Khordoc, but in accordance with the findings of conceptual metaphor theory, I will argue below that this visual convention is a non-linguistic manifestation of metaphors capturing quantity in terms of size and vertical elevation, which are said to be motivated by the experiential correlation between, on the one hand, “perception of volume and judgment of (more general sense of) quantity” (Grady 1997a: 285) and, on the other hand, “quantity and level in piles, fluids in containers, etc.” (Grady 1997a: 285; also see Lakoff and Johnson 2003 /1980/: 16).

Leaving conceptual metaphor theory aside for a moment, it seems that the best developed Peircean account of the process in which visual signs in comics become meaningful, one that is largely free of the shortcomings ascribed above to other accounts of this kind, comes from a lesser known scholar, Hubert Kowalewski, who, in a recent article titled “From icono-linguistic unity to semiotic continuity: An alternative description of semiotic repertoire of comics” (2015), proposes to characterize this process as relying on a number of semiotic moments, which consist in the establishment of indexical, iconic, and symbolic relations between a sign and its referents in consecutive semiotic cycles. While the limited scope of Kowalewski’s (2015) article prevents him from offering a comprehensive characterization of visual signs found in comics at the level of specificity situated below indexes, icons, and symbols, he does recognize the need for it, as evidenced by the comments he makes on the metaphorical iconicity of these visual signs. On the whole, Kowalewski’s (2015) account closely harmonizes in spirit with my own contribution to the intersection of comics scholarship and Peirce’s sign theory (Szawerna 2013a), in which I characterize a range of visual representations commonly found in comics as signs partaking of the three Peircean “modes of representation” (Jappy 2013: 79) — indexicality, iconicity, and symbolicity — as well as subtypes of iconicity (referred to as imaginal, diagrammatic, and metaphorical iconicity) and indexicality (referred to as reagentive and designative indexicality) in relations established with multiple referents in successive semiotic cycles. ← 16 | 17 →

But for all their advantages over the previously referenced fragmentary Peircean explorations of comics form (Magnussen 2000, Khordoc 2001, Saraceni 2003, Forceville 2005, Duncan and Smith 2009, and Cohn 2010b), the two articles (Kowalewski 2015 and Szawerna 2013a) share a significant drawback as their authors stop short of postulating an analysis of the sub-iconic properties of visual signs encountered in comics exemplifying various publication formats, generic conventions, and cultural traditions in terms of conceptual metaphor theory. As I pointed out above, this kind of analysis is advantageous because it enables the analyst to account, at least in part, for the form of many non-mimetic visual signs found in comics, which is characterizable from the perspective of this theory as motivated by entrenched conceptual metaphors, which are in turn conceived of as grounded in perceptual, social, and cultural experience. This is indeed the key advantage of the existing accounts of visual signs used in comics that are wholly or partly framed in terms of conceptual metaphor theory, such as the ones by Forceville (2005, 2011), Bart Eerden (2009), Kazuko Shinohara and Yoshihiro Matsunaka (2009), Forceville, Tony Veale, and Kurt Feyaerts (2010), Cohn (2010b), Michael Abbott and Forceville (2011), Elisabeth Potsch and Robert F. Williams (2012), and Szawerna (2014, 2016). All these accounts recognize and highlight the pivotal role of experientially grounded conceptual metaphors in providing a common basis for effective communication between creators of comics and comics readers, accomplished with the use of non-mimetic visual signs belonging to the expressive repertoire of the comics medium.

At the same time, by focusing on the metaphorical underpinnings of non-mimetic visual signs found in comics, these accounts greatly downplay the contribution of semiotic modes other than metaphorical iconicity to the shaping of the overall meaning achieved by these signs. This is undesirable because (1) non-mimetic visual signs found in comics exhibit other semiotic characteristics over and above metaphoricity, and (2) visual signs of this kind are often multiply metaphorical, and their metaphoricity is variously related to their other characteristics in the consecutive semiotic cycles, which may be either obligatory or optional. For example, the semiosis of written representations of environmental sounds, variously referred to as “sound effects” (McCloud 2006: 147), “onomatopoeic sound effects” (Duncan and Smith 2009: 145) or simply “onomatopoeias” (Duncan and Smith 2009: 156; Forceville, Veale, and Feyaerts 2010: 62), presupposes a number of obligatory as well as optional semiotic cycles involving complex interaction of Peirce’s semiotic modes, with only some of the cycles featuring metaphoricity.

At this juncture, it will be instructive to take a closer look at the process of semiosis in which one example of the kind of onomatopoeia used in comics achieves its meaning.

In one obligatory cycle of semiosis, an environmental sound, such as the sound of a gunshot, is transformed into what Kowalewski (2015: 32) refers to as the “proxy” sound, that is, a representation made up of linguistic units belonging to the phonological repertoire of the sign’s creator, such as /blæm/. In this semiotic cycle, the dominant mode of semiosis is imaginal iconicity insofar as the similarity which is observable between the sign’s representamen, that is the proxy sound, and its object, ← 17 | 18 → that is, the sound of the gunshot, depends on the acoustic/auditory properties (pitch, duration, loudness, etc.) shared by the sign’s two relata, which may be regarded as simple qualities, in the sense of Peirce (CP § 2.277), that is, “sensory qualities that do not require complex cognitive operations to be understood” (Elleström 2013: 105–106). It is in this semiotic cycle, as pointed out by Szawerna (2013a: 65) and Kowalewski (2015: 33), that the sign acquires its intuitively recognizable onomatopoeic quality. Additionally, however, the proxy sound qualifies as a Peircean symbol on account of its arbitrariness, which in turn follows from the fact that the linguistic units of which it is composed are highly conventionalized phonemes of a specific verbal language: the language of the sign’s creator.

In another obligatory semiotic cycle, the proxy sound /blæm/ is transformed into its orthographic counterpart: the written form blam. With relation to the phonological units of the proxy sound, the letters making up the sound’s orthographic representation qualify as Peircean symbols insofar as their form is arbitrary, by virtue of being in no way motivated by the acoustic/auditory characteristics of the signified phonological units, as well as conventional, in the sense that the letters belong to the fixed inventory of a writing system based on the Latin alphabet. Additionally, the letters making up the orthographic representation of the proxy sound qualify as metaphorical hypoicons of the phonological units they signify in that they capture these virtual acoustic/auditory events in the purely visual form of discrete marks on a surface. Conceived of as a whole, the orthographic form blam exhibits two kinds of iconicity with relation to the proxy sound it represents. On the one hand, it qualifies as a metaphorical hypoicon of the corresponding proxy sound because it depicts this higher-order virtual acoustic/auditory event in the purely visual form of a spatially bounded static planar representamen. On the other hand, this orthographic form is aptly characterized as a diagrammatic hypoicon of the corresponding proxy sound due to the one-to-one correspondence between the spatial arrangement of the constitutive letters of blam and the temporal progression of the phonological units making up /blæm/. Additionally, the inherent left-to-right vectorization of the orthographic form blam, determined by the prescribed reading order characterizing texts composed in any writing system based on the Latin alphabet, but not necessarily in other writing systems, confers the status of the Peircean symbol onto this orthographic representation of the proxy sound /blæm/.

In yet another obligatory semiotic cycle, the orthographic form blam functions in its entirety as a metaphorical hypoicon of the signified gunshot sound, which is understood to ring out in the world of the story, because it captures this virtual acoustic/auditory event in the purely visual form of a bounded static planar representamen. In this semiotic cycle, however, the orthographic form blam additionally exhibits two kinds of indexicality. On the one hand, this orthographic form counts as a designative index of the signified sound of a gunshot because the presence of this form inside the frame of the host panel indicates the occurrence of the corresponding environmental sound within the story-world situation depicted by the panel. On the other hand, this orthographic form qualifies as a reagentive index of the source of the visualized gunshot sound — a firearm of one kind or another — because the ← 18 | 19 → presence of this form inside the frame of the host panel presupposes the existence of such a source within the story-world situation depicted by the panel, irrespective of whether or not the firearm’s pictorial representation is found inside the panel’s frame.

In other semiotic cycles, which are not obligatory, variations in the typography of the letters making up a written representation of an environmental sound may confer additional semiotic characteristics upon this visual sign. For example, non-standard thickness and/or height of the letters making up a representation of this kind almost invariably correlate with non-standard volume of the visualized sound, in accordance with the previously invoked experientially motivated metaphors capturing quantity in terms of size and vertical elevation. Also, the shape of the letters making up a written onomatopoeic form may be indicative of certain acoustic/auditory properties of the sound it represents. McCloud (2006: 147) points out that in orthographic representations of environmental sounds, specific acoustic/auditory properties of the signified sounds are visualized in different ways. For example, the sound of a gunshot, the roar of a jet engine, the cracking of a whip, etc., are often represented by means of an onomatopoeic form written in sharp-edged, angular letters. Such orthographic representations may be regarded as metaphorical hypoicons of environmental sounds perceived as unpleasant to the ear (loud, shrill, strident, etc.), in accordance with a synaesthetic metaphor, in the sense of Yeshayahu Shen (2008: 302–305), whereby an auditory property of an environmental sound in the target domain (specifically, its perceptual unpleasantness) corresponds to a tactile property of a physical object in the source domain (specifically, its sharpness).

This account of the semiotic process in which a written onomatopoeic representation of an environmental sound becomes meaningful may not be exhaustive, but it suffices to support the main methodological thesis of this study, according to which metaphoric iconicity of non-mimetic visual signs regularly found in comics is most productively explored with reference to its experiential basis, which motivates many formal properties of such visual signs, and in the context of the overall semiotic makeup characterizing these non-mimetic visual signs because this type of experientially motivated iconicity interacts with other semiotic modes, and together they build and shape the meaning of these signs in a series of semiotic cycles. This thesis determines the principal goals of this study, which consist in (1) integrating elements of Peirce’s theory of signs with elements of conceptual metaphor theory into a unified cognitive-semiotic approach to the study of metaphorically motivated non-mimetic visual signs found in comics and (2) implementing this approach by formulating a broad-based cognitive-semiotic characterization of the way in which a range of non-mimetic visual signs regularly used by creators of comics representing various publication formats, generic conventions, and cultural traditions to depict situations, motion events, sound events, and psychological experiences become meaningful.

Insofar as the proposed approach to the analysis of metaphorically motivated non-mimetic visual signs found in comics integrates elements of Peirce’s theory of signs with elements of conceptual metaphor theory — a quintessential cognitive ← 19 | 20 → linguistic framework (Grady 2007: 188) — it may be considered an example of the kind of theoretical fusion between semiotics and cognitive linguistics in the area of research into visual signification that was postulated a while back by Alina Kwiatkowska (2011), who argued that the semiotic makeup of visual representations may be productively explored from the cognitive linguistic perspective for the reason that cognitive linguistics, understood as a linguistic paradigm which subsumes an array of distinct, albeit overlapping, theories and research programs (van Hoek 1999: 134; Evans and Green 2006: 3), views the principles of linguistic organization as motivated by the principles of general cognition. Insofar as most theoretical and descriptive constructs developed by cognitive linguists are not regarded as being uniquely verbal, but are instead conceived of as pertaining to cognition in general, their application to the analysis of non-linguistic signs, including a broad and varied range of visual representations, is entirely justified. As I pointed out it in an earlier publication (Szawerna 2013b: 139–140), a number of theoretical concepts and descriptive instruments developed by cognitive linguists may be applied to the characterization of non-linguistically encoded meanings, and the broad applicability of these constructs follows from a central tenet of cognitive linguistics whereby the meanings symbolized by linguistic expressions are shaped by the same mechanisms that guide non-linguistic cognition — most notably, by metaphor, metonymy, and conceptual integration. Understood in this way, these general cognitive mechanisms appear as basic meaning-making mechanisms which are instrumental in the creation and interpretation of verbal as well as non-verbal signs.

Owing to the fact that this study of metaphorically motivated non-mimetic visual signs found in comics is framed in terms of conceptual metaphor theory,4 it qualifies as an exploration in what has come to be known as “multimodal linguistics” (Bateman 2008: 38) — a burgeoning scholarly field whose proponents call upon linguistic theory in their efforts to characterize the meaning-making potential of manifold sign complexes, which partake of one or more “semiotic modes” (Bateman 2008: 38), such as the gestural mode, the pictorial mode, the sonic mode, and the musical mode, to realize this potential.5 These sign complexes, sometimes referred to as “modal ensembles” (Bezemer and Kress 2016: 6), may take the form of long-lasting “semiotic artefacts” (Bateman 2008: 38), such as, for example, photographs, advertisements, cartoons, comics, and films, or they may take the form of transitory, or rapidly fading (Hockett 1963: 7), products of multimodal interaction, such as Cornelia Müller and Alan Cienki’s “verbo-gestural utterances” (2009: 300).

Given that conceptual metaphor theory constitutes a prominent theoretical framework of “the cognitive linguistics enterprise” (Evans and Green 2006: xix), this ← 20 | 21 → study of metaphorically motivated non-mimetic visual signs found in comics may be characterized more specifically as being situated at the crossroads of multimodality studies and cognitive linguistics, sometimes referred to explicitly as “multimodal cognitive linguistics” (Langlotz 2015: 55; Moya Guijarro 2015 /2013/: 117), which, according to María Jesús Pinar Sanz (2015 /2013/: 2), constitutes a major linguistic approach to analyzing modal ensembles, along with the approaches founded on systemic functional grammar (O’Toole 1990; Kress and van Leeuwen 2006 /1996/; Bezemer and Kress 2016) as well as discourse analysis and sociolinguistics (Norris 2004).6 It seems that researchers working in the area of multimodal cognitive linguistics have so far tended to draw on the interrelated frameworks of conceptual metaphor theory, initiated by George Lakoff and Mark Johnson (2003 /1980/), image schema theory, originally formulated by Johnson (1987), and conceptual integration theory, also known as blending theory, articulated most fully by Gilles Fauconnier and Mark Turner (2002), in numerous explorations of the way in which diverse modal ensembles, including musical pieces, paintings, sculptures, advertisements, cartoons, and films, acquire meaning. This tendency is observable across a range of publications, including the studies written by Lawrence Zbikowski (2002), Cienki and Müller (2008), Maarten Coëgnarts and Peter Kravanja (2012), Elżbieta Górska (2012, 2014a, 2014b), and Kwiatkowska (2013), but also multiple contributions to the volumes edited by Agnieszka Libura (2007), Forceville and Eduardo Urios-Aparisi (2009), Pinar Sanz (2015), and Coëgnarts and Kravanja (2015).

The tendency on the part of multimodal cognitive linguists to draw on the interrelated frameworks of conceptual metaphor theory, image schema theory, and conceptual integration theory is also noticeable at the intersection of multimodal cognitive linguistics and the new comics scholarship.

Pinar (2014) shows how the conceptual metaphor LIFE IS A JOURNEY is visually exemplified in a comic book by Raymond Briggs, Forceville (2011) calls upon conceptual metaphor theory with a view to characterizing the meaning-making potential of the pictorial runes found in a classic Tintin album by Hergé, and Forceville (2005), Eerden (2009), Shinohara and Matsunaka (2009), and Abbott and Forceville (2011) employ the instruments of conceptual metaphor theory in their explorations of the ways in which emotions experienced by characters in European and Japanese comics are represented in the form of conventionalized visual signs. In turn, Forceville, Veale, and Feyaerts (2010), Potsch and Williams (2012), and Szawerna (2014, 2016) combine elements of conceptual metaphor theory with elements of image schema theory in their analyses of conventionalized visual signs used by creators ← 21 | 22 → of comics to depict sound events, motion events, and psychological experience. Last but not least, Cohn (2010a) invokes conceptual metaphor theory and the theory of conceptual integration in an account of how the sequential processing of panels making up short comic strips triggers off the interpretation of the pictures and writing encapsulated by these panels in terms of conceptual metaphor, metonymy, and conceptual integration, while Szawerna (2012a) employs the instruments of conceptual integration theory with a view to characterizing selected elements of the story world presented in Alan Moore, Dave Gibbons, and John Higgins’s graphic novel Watchmen (1986–1987) as conceptual blends.

But an exploration of conventionalized visual signs belonging to the standard repertoire of the comics medium, such as the one offered in this study, might be considered linguistic in another sense — as a principled examination of linguistic signs — provided that the expressive resources available to creators of comics were recognized and theorized as a language. This is the position taken by Cohn in his book The Visual Language of Comics: Introduction to the Structure and Cognition of Sequential Images (2013b), in which he argues that the titular notion of the visual language of comics is meant to be taken literally, “on par with verbal and signed languages” (Cohn 2013b: 4). Cohn’s (2013b) view of the relation between natural languages and the expressive resources of comics embeds his study in a broader strand of visual semiotics — one extending far beyond the necessarily limited scope of contemporary comics studies — which is guided by the “genuine belief that all meaning is similar to the linguistic kind” (Kwiatkowska 2011: 315). Whether or not Cohn’s (2013b: 4) position may be adopted as an assumption underlying this linguistically oriented contribution to the new comics scholarship depends on the validity of his argument, evaluated from the perspective of contemporary linguistics in general and from the perspective of multimodal cognitive linguistics in particular. In the following few paragraphs, I will summarize Cohn’s (2013b: 4–7) argument, evaluate its validity, and decide whether or not Cohn’s (2013b: 4) position is compatible with the cognitive-semiotic approach advanced in this study.7

In his book, Cohn (2013b: 4–7) claims that the set of non-verbal expressive resources utilized by creators of comics qualifies as a natural language insofar as it exhibits three characteristics which are individually necessary and collectively sufficient to distinguish languages from other “human behaviors” (Cohn 2013b: 7), such as music, dance, athletic skills, gestures, etc. The first of these characteristics is modality, which concerns the manner in which signs belonging to a set of expressive resources are produced. Cohn (2013b: 4) explains that there are three modalities pertaining to languages: “creating sounds from the mouth (verbal), moving the body (manual/bodily), or making marks on a surface (visual-graphic).” The second characteristic is meaning, conceived of as “explicit” (Cohn 2013b: 5) concepts ← 22 | 23 → which “may be abstract or concrete” (Cohn 2013b: 4). The last characteristic is grammar, defined as “a system of rules and constraints for sequential expressions of meaning” (Cohn 2013b: 4). Cohn’s (2013b: 4–7) characteristics are binary in that they are conceived of as either applicable or non-applicable to a given behavior, with no middle ground between the two values. Cohn (2013b: 7; Table 1.1) draws a thick dividing line between verbal, signed, and visual languages, which are jointly characterized by modality, meaning, and grammar, and non-linguistic behaviors, which are characterized by modality, but lack meaning (music, dance, athletic skills), grammar (gestures; meaningful images, paintings, diagrams, etc.), or both meaning and grammar (abstract art).

One problematic characteristic of the set of expressive resources referred to by Cohn (2013b) as the visual language of comics concerns the set’s capacity, alleged by Cohn (2013b: 4), to express explicit meanings of varying degrees of abstractness, which may be referred to using Charles F. Hockett’s (1963: 8) term “semanticity.” While it is certainly true that natural languages exhibit this kind of semanticity insofar as they are used on a regular basis to express a broad range of explicit meanings which vary in their degree of abstractness depending on whether they relate to the more tangible aspects of human experience, as in example (1a), or the less tangible aspects of it, as in example (1b), the kinds of meanings that may be expressed in Cohn’s (2013b) visual language of comics are invariably of the former kind.

(1a) When Jerry came into the playroom, he sat down, took off his shoes and socks without help, and climbed into the sand box. (Axline 1974 /1969/: 117)

(1b) Capital inflows rose sharply and rapid growth of private credit boosted demand and spurred above-trend aggregate growth. (Stone et al. 2009: 34)

A moment’s reflection suffices to realize that the sequence of events narrated in example (1a) is readily representable in the form of a wordless comic, that is, a comic created solely with the use of the expressive resources referred to by Cohn (2013b) as the visual language of comics, while the sequence of events narrated in example (1b) is not. The reason why this is so is that unlike the events narrated in example (1b), the events of example (1a) constitute visually perceivable scenes which are straightforwardly captured by means of visual signs exhibiting imaginal and diagrammatic iconicity, that is, signs sharing simple qualities (for example, shape, size, and color) and structural relations with their referents, such as the pictorial representations regularly utilized by creators of comics, which by and large “look like the thing they represent” (Miodrag 2013: 9).

This simple example demonstrates that imaginal and diagrammatic iconicity, which constitute the dominant modes of representation characterizing comics (Jappy 2013: 9), greatly constrain the semanticity of the set of the non-verbal resources recognized and theorized by Cohn (2013b) as the visual language of comics, which are ideally suited to reproducing the human experience of vision by capturing ← 23 | 24 → visually perceivable scenes.8 Insofar as the semanticity of the set of non-verbal expressive resources utilized by creators of comics is a lot more limited than the semanticity of natural languages, the former cannot really be considered, pace Cohn (2013b: 4), to be on a par with the latter with regard to the kinds of meanings it can express.

What makes the semanticity of natural languages virtually unlimited are certain properties of their grammar, which, characterized from the cognitive linguistic perspective, constitutes “a structured inventory of conventional linguistic units” (Langacker 1987b: 57) of the phonological, semantic, and symbolic kind (Langacker 1987b: 73). Conceived of in this way, the grammar of a natural language, such as English, specifies how conventional linguistic units function as parts of more inclusive units of this kind. This is how Ronald W. Langacker (1987b: 73) illustrates his conception: “For example, the phonological units [d], [ɔ], and [g] function as components of the higher-order phonological unit [[d]-[ɔ]-[g]]. This in turn combines with the semantic unit [DOG] to form the symbolic unit [[DOG]/ [[d]-[ɔ]-[g]]], to which we can add the plural morpheme to obtain a higher-order symbolic unit, and so on.” Langacker’s (1987b: 73) example shows how the grammar of a natural language is founded on the principles referred to by Hockett as “discreteness” (1963: 8) and “duality of patterning” (1963: 9), whereby various combinations of discrete phonological units belonging to a limited set are paired with units of meaning to make up minimal symbolic units, referred to as morphemes, which in turn combine with other symbolic units to make up more and more inclusive symbolic units at progressively higher levels of linguistic organization. The grammar of a natural language is based on another principle, dubbed “arbitrariness” by Hockett (1963: 8), whereby typically there is no motivation, such as, say, similarity, for the pairing of a phonological unit with a semantic unit as the two “poles” (Langacker 1987b: 91) of a morpheme. Together, discreteness, duality of patterning, and arbitrariness make the semanticity of natural languages, both verbal and signed,9 virtually unlimited insofar as they allow language users to pair meanings of any kind, irrespective of their degree of abstractness, with various combinations of phonemes in an arbitrary fashion.

Importantly for my considerations, the set of expressive resources referred to by Cohn (2013b) as the visual language of comics does not exhibit discreteness, duality of patterning, or arbitrariness insofar as it does not employ a small fixed set of discrete meaningless units akin to the phonemes of natural languages that could be combined in various ways and then arbitrarily paired with explicit meanings. ← 24 | 25 → Even the smallest “graphic schemas” (Cohn 2013b: 28) deployed by creators of comics, such as the ones for eyes, feet, ears, noses, etc., presented by Cohn (2013b: 29, Figure 2.3), qualify as signs which are non-arbitrary in that they are motivated iconically by the imaginal and diagrammatic similarity they bear to their referents as well as “dense” (Mitchell 1986: 67) in that they cannot be resolved into discrete meaningless units belonging to a limited inventory. That is why, as Miodrag (2013: 9) rightly points out, the meanings expressed by non-verbal signs encountered in a newly read comic are for the most part readily interpretable, while the meanings of newly encountered morphemes have to be explicated. Since discreteness, duality of patterning, and arbitrariness do not seem to characterize the set of non-verbal expressive resources utilized by creators of comics, it cannot be argued to possess the kind of grammatical organization featured by natural languages and, consequently, cannot be considered, pace Cohn (2013b: 4), to be on a par with such languages in terms of grammaticality.

It appears that the qualitative differences observable between natural languages and the set of expressive resources referred to by Cohn (2013b) as the visual language of comics with regard to their semanticity and grammaticality, the defining characteristics of language invoked by Cohn (2013b: 4) himself, effectively invalidate Cohn’s (2013b: 4) explicitly stated position whereby the set of non-verbal expressive resources utilized by creators of comics is a fully articulated natural language, “on par with verbal and signed languages.” Given the said differences, it seems unlikely that verbal languages, signed languages, and the set of non-verbal expressive resources utilized by creators of comics make up a uniform category of natural languages, which may, as Cohn (2013b: 7) wants, be sharply distinguished from “human behaviors” which do not qualify as natural languages because they cannot express explicit meanings and/or lack grammar. But Cohn’s (2013b: 4) choice of the defining characteristics of language seems somewhat arbitrary, given that natural languages and the set of non-verbal expressive resources utilized by creators of comics may be constructively compared and contrasted not only in terms of semanticity and grammaticality, but also in terms of such “design-features” (Hockett 1963: 6) as “broadcast transmission” (Hockett 1963: 7), “rapid fading” (Hockett 1963: 7), “interchangeability” (Hockett 1963: 7), and “reflexiveness” (Hockett 1963: 10).

Details

Pages
487
Publication Year
2017
ISBN (PDF)
9783631723760
ISBN (ePUB)
9783631723777
ISBN (MOBI)
9783631723784
ISBN (Hardcover)
9783631675212
DOI
10.3726/b11165
Language
English
Publication date
2017 (June)
Keywords
Visual semiotics Multimodality Conceptual metaphor Visual narratives Signification Iconicity
Published
Frankfurt am Main, Bern, Bruxelles, New York, Oxford, Warszawa, Wien, 2017. 487 pp., 66 b/w ill.
Product Safety
Peter Lang Group AG

Biographical notes

Michał Szawerna (Author)

Michał Szawerna is Assistant Professor at the Institute of English Studies, University of Wrocław. His research interests include visual signification, cognitive linguistics, and Peirce’s sign theory.

Previous

Title: Metaphoricity of Conventionalized Diegetic Images in Comics