The Representation of Affect in Chinese Visual-Verbal News Commentary: A Corpus-Based Study

Visual-verbal discourses are often loaded with affect which is one of the three semantic types of attitude in Appraisal Theory. After corpus collection and annotation on the basis of 100 cases of Chinese new commentary, this research conducts a quantitative study of the representation of affect in Chinese visual-verbal news commentary, which reveals that 1) Affect is not common in the corpus and the most frequently occurred affect is the kind of affect from the depicted disadvantaged participant. 2) While authorial and third-party affects are mainly negative affect toward negative behaviors, affects from the depicted disadvantaged participant mainly express their negative emotion toward the poor encounter, and affects from the depicted negative participant mainly show their negative emotion toward positive behaviors. 3) As for the types of semiotic modals, the majority of affects are verbal affects. Besides, there are both visual and verbal affects from the depicted participants, but neither visual authorial nor third-party affect is found. Findings above prove that the visual-verbal representation of affect in Chinese visual-verbal commentary is a complex process of systematic choices-making of different representational factors, and these choices exactly contribute to the construction of such genre.

in/security, dis/satisfaction, dis/inclination and not/surprise [1] [2] [3]. As one of the most common forms of multimodal discourse, visual-verbal discourses are usually loaded with affect of these different kinds. However, compared to studies of verbal affect, the analysis of affect in visual-verbal discourse has not attracted attention until recently, and is mainly discussed together with the other two kinds of attitude, i.e., judgment and appreciation. Economou conducts a system of visual appraisal based on the analysis of verbal-visual news media texts and finds that inscribed attitude in corpus is mainly related to primary human emotions [4] [5] [6].
Bednarek & Caple undertake a case study of an on-line multimodal reporting, which illustrates that news values were constructed by choices in language and image [7]. Pounds employs Appraisal Theory and analyzes the multimodal expression of authorial affect in a British television, which reveals that authorial affect can be conveyed through the audio-visual and dialogic dimensions [8]. As for researches on visual/verbal resources for the representation of attitude, Hood summarizes four different kinds of explicit verbal resources [9]. Martin & White exemplify how adjectival and the selection of ideational meaning can attribute to affect [1]. Bednarek provides a list of common English emotion terms [3]. Liu classifies eight different structural potentials of affect in English [10]. Lemke summarizes that there were 8 types of visual/verbal resources for evaluative meaning in political cartoon [11]. Martinec analyzes how affect could be expressed through actions [12]. Forceville discusses the visual representations of the idealized cognitive model of anger, one typical primary affect [13]. Feng & O'Halloran present how the semiotic resources of facial expression, touch and body orientation can be used to represent emotive meaning [14].
Swain exemplifies that visual affect is easily inscribed in the faces, gestures and body postures of depicted human participants [15]. These studies gave us a better understanding of visual/verbal resources for representing affect and other attitudes. However, few has tried to make use of a Mandarin Chinese visual-verbal corpus to undertake a quantitative analysis of the representation of affect in such discourse, which leaves large space for further investigation. Considering this, this study endeavors to examine the representational characteristics of affect in Chinese visual-verbal news commentary based on a quantitative analysis of the corpus so as to provide a possible analytical modal of the quantitative examination of multimodal attitude in general including affect, judgment and appreciation, especially in Chinese corpus. Open Journal of Social Sciences stance toward a certain negative social phenomenon. In this study, a distinction between macro-attitude and micro-attitude (and corresponding, between macro-affect and micro-affect) is made. The former refers to the overall authorial stance, the latter refers to the specific attitudes that can be identified in specific verbal expressions or in the cartoon. Generally speaking, the macro-attitude is realized by the designing of specific micro-attitudes, among which there are a lot cases of micro-affect. From the perspective of Systemic-Functional Linguistics, affect, like other kinds of attitudinal meaning, is not randomly realized in discourse, either. On the contrary, generally the representation of affect can be regarded as a complex process of systematic choices-making of different representational factors, which involves at least five aspects as followed: 1) Affect host. Pre-sample analysis shows that in general, there are two types of affect host in Chinese visual-verbal new commentary, i.e., the author and the third party on the one hand, and the depicted participants on the other hand.
The latter further consists of two parts, the disadvantaged participant such as the victims of a certain event, etc. and the negative participant such as a bribe, etc.
2) Affect target. Nine types of affect target were found in the corpus: the negative behavior, the negative thing, the negative figure, the negative value, the positive behavior, the positive thing, the positive figure, the positive value, and the poor encounter of the disadvantaged participant.

Representational Resources
In Chinese visual-verbal new commentary, the representation of affect makes use of both verbal and visual resources. On the one hand, Martin & White (2005: 58) pointed out that the canonical grammatical realization for attitude is adjectival. Actually, other content words such as attitudinal nouns like 热情 (rè qíng, ambition), verbs like 钟情 (zhōng qíng, deeply love) and adverbs like 快快乐 乐地 (kuài kuài lè lè, happily) are also potential explicit resources. In other words, explicit verbal representational resources are attitudinal content words.

Corpus Annotation
The corpus was systematically annotated to assist further quantitative analysis of the data. To begin with, an initial processing, putting every single tar- get visual-verbal discourse into an individual document, was undertaken, which was followed by an initial identification of micro-attitude. Any clause in which micro-affect was identified was put into a single line. And for those clauses in which more than one micro-affect were identified, clauses were repeatedly listed in a single line by corresponding times. The same processing is applied to cartoons. Second, the labels "TITILE-text", "TITILE-cartoon", "BALLOONS-talk"

Case 1: 唯独忽略其中最为重要的一点，那就是还孩子们应有的快乐童年。
Only what is neglected is the most significant part, that is, children should be given back their deserved happy childhood.     more common compared with those from the author and the third party. This statistical result is actually not difficult to understand, either. For one thing, the visual representation of affect from the depicted sides provides the readers with a reproduction of the event without hurting the degree of narrative objectivity of the discourse. However, the objectivity of authorial emotive evaluation is easily expressed if too many cases of authorial and third-party affect were visually expressed, which further breaks the degree of reliability of authorial stance. Table   3    The mother in the family suffers from a fatal disease and is anxious about being uncapable of paying the medical cost.

Detailed Analysis of the Visual-Verbal
Cases of affect-insecurity are all verbal affect. For example, in Case 3, the use of the attitudinal adjective 发愁 (fā chou, anxious) explicitly represents the affect of insecurity of the depicted disadvantaged, that is, the badly sick mother who worries a lot about how to pay the huge medical cost. Cases of affect-unhappiness were found both in visual and verbal corpora. For instance, Case 4 below expresses the depicted disadvantaged farmer's strong unhappiness by the use of the attitudinal adverb 难过得 (nán guò de, sadly).

Case 4: 济南那位农民六亩田的卷心菜蚀大本，难过得喝农药寻了短见。
The six acres of cabbage was a great loss for the farmer from Jinan, who sadly killed himself by drinking pesticide.
Cases of affect-disinclination were all verbal ones. Take Case 5 below for instance. The difficulty of getting medical service is still quite huge for many ordinary people in China. The affect-disinclination of Gao Yan, the depicted disadvantaged working-class father whose daughter was suffering from severe illness, is easily represented through the attitudinal adverb 只好 (zhǐ hǎo, have no choice but to). There are both visual and verbal cases of affect-surprise. Case 6 is from the same discourse as that of Case 5. The father Gao Yan and his wife were quite unexpected and shocked (the affect-surprise) when they noticed that there were lots of people who were waiting for check-in before them. Similarly, this emotive evaluation was represented by the attitudinal adverb 没想到 (méi xiǎng dào, unexpectedly).

为了尽快让女儿就诊，高岩只好多花钱。
To have his daughter's medical treatment begin as soon as possible, Gao Yan had no choice but to spend more money.

Case 6:
高岩夫妻俩早七点赶到北京儿童医院去挂号，没想到人山人海。 Gao Yan and his wife got up early around 7 to 8 am to check in the Beijing Children's Hospital, but unexpectedly they found that there were a crowd of people.
In Case 7 (Figure 1), there are two depicted participants, one left boy who was laughing and one right boy who lowered his head and showed a sad face with frowned eyebrows, a typical embodied emotive behavioral process. Take its verbal context into consideration, this process explicitly represents the boy's affect-unhappiness, i.e., he was sad thinking that his father was not as rich as fathers of other children. In Case 8 (Figure 2), the depicted disadvantaged man, that is, the old grandfather on the left, is equipped with a widely opened mouth, which is an ideological behavioral process, expresses the grandfather's strong surprise (affect-surprise) he felt when he realized how difficult it was to check in a good hospital for medical service.

Representation of Affect from the Depicted Negative Participant
Among the 46 cases of affect from the depicted negative participant, there are 7 cases of visual affect and 39 cases of verbal affect. As far as the type of affect target is concerned, more than 36% of the cases are affect toward the positive behavior. Examination of these cases demonstrates that they mainly express the inappropriate negative affect of the negative participant toward good behaviors.
Take Case 9 and Case 10 for example.

Case 9: 而获救者吝于言谢，给这起悲剧画上黯淡的结尾。
However, the rescued man was too mean to express his gratitude, which drew a dark end of that tragedy (the saver drowned himself).   Similarly, the negative emotive attitude of the public, i.e., the third party, is represented by the expression of 公众愤慨 (gong zhòng fèn kǎi, the public are indignant).

Case 13: 更令我们心痛的是，两个青年死亡六天了，没有人出来公开道歉。
What makes us heartbroken is that no one came out to apologize even six days after the death of the two young man.

Conclusions
This study addresses the issue of how the attitude of affect is visually and verbally represented in the corpus of Chinese news commentary. A thorough quantitative analysis reveals that the visual-verbal representation presents three main characteristics. First, compared to judgment and appreciation, the frequency of occurrence of affect is low and the most frequently occurred affect is the affect from the depicted disadvantaged participant. Second, while authorial and third-party affects are mainly negative affect toward negative behaviors, affects from the depicted disadvantaged participant mainly express their negative emotion toward the poor encounter, and affects from the depicted negative participant mainly show their negative emotion toward positive behaviors. Third, the majority of affects are verbal affects. While there are both visual and verbal affects from the depicted participants, no visual authorial or third-party affect is found.
Within the theoretical framework of Systemic Functional Linguistics, the representation of affect can be regarded as a complex process of systematic choices-making of different representational factors. Actually, the findings above exactly demonstrate such process. For one thing, the design of authorial and third-party affect directly manifests the macro-attitude (the writer's critical attitude toward the issue under discussion), which is effective in building a convincing argument and creating possible resonation between the writer and the readers. In the meantime, the degree of objectivity of the argument is not sharp comparison of this affect and the macro-attitude, which helps constructing the rationality of the latter. As a corpus-based study of the visual-verbal representation of affect, this paper paves the way for the researchers to further investigate the visual-verbal representation of other kinds of attitude rather than affect as well as the role of visual-verbal interaction in attitude representation. In addition, this paper also indicates that designers and writers of multimodal news commentary should pay special attention to the integrated use of both verbal and visual materials so as to effectively arise critical resonance on the side of readers and makes the discourse as persuasive as possible.