Mood assessment via animated characters: An instrument to access and evaluate emotions in young children ()
1. INTRODUCTION
Mood and anxiety disorders are prevalent mental health problems that often begin in the preschool or early school years [1]. Their long term impact on development can be severe (risk of academic and career impairments, family dysfunction, social isolation, substance abuse) [1, 2]. Therefore, it is imperative to accurately assess young children with these problems to ensure early intervention.
The ability to identify, label, and discuss one’s own feelings is an important mental health skill. Termed “emotional competence” or “emotional understanding”, this ability predicts later academic competence in preschoolers, and its absence has been linked to parentand teacher-reports of mental health symptoms in this age group [3,4]. Emotional responses are largely physiological in infants, but become linked to specific cognitions and actions with experience and development [5]. Emotional understanding develops gradually, beginning with basic emotions (e.g., anger, fear, surprise, disgust) in very young children, then expanding to include social emotions (e.g., shame, jealousy), and finally including subtle distinctions between emotions of similar valence (e.g., for positive valence: distinguishing surprised versus happy) [6].
Researchers in the development of emotions and cognitive behavioral theorists have both linked children’s internalizing psychopathology to learned, maladaptive connections between emotions, cognitions, and actions [5,7,8]. For example, children with anxiety disorders often assume a situation is dangerous and try to avoid it if they have experienced an unpleasant degree of emotional arousal in the situation in the past, even if the situation is not truly dangerous. Overcoming their anxiety involves not only exposure to the feared situation but also developing adaptive cognitions and actions in response to anxious feelings. Unfortunately, many of these children struggle to identify and label their feelings [9,10] (i.e., show low emotional competence), resulting in continued maladaptive cognitive and behavioral responses, and continued symptoms. Helping these children accurately label and discuss their feelings is thus a crucial aspect of both assessment and intervention.
Unfortunately, psychological assessment of children in this age group is laden with difficulty (reviewed in [11]). Most well-validated instruments for eliciting internalizing symptoms target children ages 8 and above [1,11]. Even if administered orally, such measures may not be valid in younger children due to their limited vocabulary for feelings [11,12]. Therefore, clinicians often rely on interviews of the child (sometimes assisted by play materials), but still find that young children struggle to communicate internally experienced affective states [9, 10]. Consequently, assessment of feelings often relies on parentor teacher-reports, even though reports from such informants generally correspond poorly to the child’s self-reports [13,14]. Furthermore, by relying on them we miss the opportunity to foster the child’s ability to communicate about feelings. This ability is a prerequisite for effective psychotherapies (e.g., cognitivebehavioral therapy) [7]. A few static pictorial scales for internalizing symptoms exist (e.g. Koala Fear Questionnaire) [15], but static images have been found to be less recognizable and less preferred than animated images by young children [16].
Animation-based instruments offer a promising option for assessing young children’s feelings, with limited reliance on language. Due to a century of exposure to animated films, the ability to “read” character-based animation is a trait shared by people of all cultures and ages [17]. An animated instrument may be uniquely appealing to young children and may prompt discussion of feelings in a nonthreatening way. Moreover, preschoolers identify both basic and social emotions more accurately when given situational cues (as one would in an animation) than when presented with emotion-specific faces [6]. An animated instrument would not replace diagnostic interviews. Rather, like questionnaire measures in older children, it would allow a qualitative and quantitative description of negative feelings from the child’s point of view, and a potential means of evaluating therapeutic change.
We developed such an instrument, “Mood Assessment via Animated Characters” (MAAC), which uses computerized animations to aid clinicians in assessing children’s feelings [18]. In a pilot study (separate sample), MAAC helped clinicians engage children in discussions of their feelings and showed promise in distinguishing children with anxiety disorders from nonanxious children [18]; children with anxiety disorders explored fewer emotions spontaneously, rated positive emotions lower, and visited positive emotions later. These findings are consistent with literature reporting heightened caution and heightened attention to threatening stimuli (in this case, negative emotions) in anxious individuals [19,20].
To ensure appropriate clinical use of MAAC, however, we gathered further data on its psychometric properties in a larger sample of anxious and nonanxious children and refined our analytic methods. We were particularly interested in including a larger number of young children (i.e., those with whom clinicians are most likely to utilize MAAC), in determining if MAAC’s many specific animations clustered into clinically meaningful factors, and in discerning which aspects of MAAC best distinguished clinical and nonclinical groups. This is an important first step towards validating the instrument, as samples of young children with other diagnoses would be needed to fully understand how to best apply MAAC in clinical practice. We hypothesized that 1) responses on the 16 emotions included in MAAC would cluster into clinically relevant factors (face validity); 2) consistent with previous findings, anxious children’s feelings ratings and order of feeling selection would differ from those of nonanxious children, and anxious children would explore fewer emotions spontaneously than nonanxious children (discriminant validity); 3) MAAC ratings of negative or fearful emotions would relate to scores on established anxiety and depression measures for children old enough to complete those measures (convergent validity). We also examined ageor gender-related differences in MAAC responses. These were exploratory analyses with no a priori hypotheses, though previous studies suggest that emotion recognition improves with age in nonclinical children [21,22], and that nonclinical preschool and school-aged girls may have a more accurate emotional understanding than their male counterparts [23,24].
2. METHOD
2.1. Subjects
MAAC was administered to 157 children ages 4 - 10 years in an outpatient Anxiety Disorders Clinic of a tertiary treatment center. At telephone intake, all parents of clinic attendees who were likely to meet study criteria were offered an extended assessment which included the MAAC research procedures. Families who agreed (approximately 80%) were compensated for the cost of a meal at the hospital (necessitated by the length of the assessment) and children were provided a toy or movie pass (their choice) as a token of appreciation. Clinical subjects (n = 74; 33 boys, 41 girls) had a primary (most impairing) Axis I diagnosis of Separation Anxiety, Generalized Anxiety, or Social Anxiety on the Anxiety Disorders Interview Schedule (ADIS) [25], a semi-structured diagnostic interview based on DSM-IV criteria and covering all major psychiatric diagnoses. Thirty-nine clinical subjects had additional diagnoses, mostly anxiety disorders (n = 30) but with ADHD (n = 9), ODD (n = 2), and tic disorders (n = 2) also represented. All ADIS interviewers were either child psychiatrists, or psychologists trained to reliability on this instrument, and with at least 3 years experience using it in research. Children taking long-acting psychoactive medications that could not be discontinued were excluded, as were children suffering from active psychosis or having sensory or medical impairments precluding their use of MAAC. Other comorbidities were allowed. Controls (n = 83; 40 boys, 43 girls) of comparable age were recruited via flyers posted in the community from which clinical cases were drawn, and had to be free of any Axis I diagnosis on the ADIS. Demographic characteristics of clinical and control subjects are summarized in Table 1, including parent reports of possible language and learning difficulties (not formally tested).Groups did not differ in age, gender, family composition, ethnicity, or parental education.
2.2. Procedure
Informed consent was obtained from parents and assent from children, in accordance with our Institutional Review Board. All children and parents completed procedures during a single day, beginning in the morning and concluding in the early afternoon. Parents completed a demographic questionnaire about their families (Family and Household Form from the Ontario Child Health Study) [26,27]. Children completed all procedures with a research assistant blind to diagnosis, who helped with reading of questionnaires if needed, and provided breaks between measures if a child was tired or uncomfortable.
Table 1. Sample characteristics by group.
No children found the procedures distressing, and several spontaneously reported that MAAC was “fun”. All children completed MAAC concerning their current feelings first, to avoid biasing responses due to exposure to other emotion-focused stimuli, and then concerning feelings in the past 2 weeks. Older children (8 - 10 years, n = 84:53 clinical, 31 control) then did standardized questionnaires related to internalizing disorders (see below), which were counterbalanced to minimize order effects. Younger children (4 - 7 years, n = 73) did the Koala Fear Questionnaire [15], the only measure applicable to this age group. Including ratings with respect to both current feelings and feelings in the past two weeks allowed for comparison with both stateand trait-focused questionnaires. At the conclusion of the assessment (or research session in controls), the responsible clinician provided each child and family with a summary of findings and recommendations based on all interviews and measures, and an opportunity to discuss their experiences as research participants. All research data were kept in locked filing cabinets and password-protected computers accessible only to the research team, consistent with procedures mandated by our IRB.
2.3. Measures
MAAC (technical features) [28,29]. Computer software and computer animations were created that display an appealing tableau of sixteen feeling states, ensuring coverage of primary emotions often cited in the literature (joy, surprise, anger, sadness, fear, disgust) [6,30,31], a balance of positive, negative, and neutral emotions, and a balance of more intense and less intense feeling states. Design considerations for user interface included: physical size of the animated character, number of characters visible on the screen at one time, duration of each animation, color palette, and method of browsing the tableau. Design considerations for characters included: gender, ethnicity, realism of the drawing (photorealistic vs. cartoon-like) [32,33], color palette, and speed of acting.
Decisions were made by consensus among the clinical and animation team, with repeated input from children in their practices and acquaintanceships. Qualitative responses to early versions of the main character (of various ages and appearances) were elicited first from clinical and nonclinical children, and each child was asked to pick a favorite. We then asked children to suggest improvements to the overall favorite character, and made further refinements. Then, animations were created representing the sixteen selected feeling states, and children were asked how they thought the character was feeling in each animation. Unclear or ambiguous animations were further refined until clear, using an iterative, user-centred approach. The main character’s gender (female) was chosen to be nonthreatening to young, possibly anxious children. Her ethnicity (Caucasian) is the most common in our clinical population but could be altered in future iterations. Making her a teenager allowed for some amusing animations (given the lanky, awkward body habitus of teens), and she also resembled a popular cartoon heroine, making her a favorite among children. Hardware included a computer with a touchscreen interface to allow children to interact directly with the graphical images using a stylus (Fujitsu-brand Tablet PC). Since completion of this study, an updated version has been made available online at https://www.sickkids.ca/ MAAC/.
Children followed standardized instructions to navigate through MAAC’s tableau (see Figure 1) of 16 emotion-specific static facial pictures of the female character, selecting ones to see animated “live”. Specifically, they were asked: 1) to indicate how they were feeling; 2) to pick a static picture of the character that matched that feeling; 3) to watch the animation of that feeling (which plays when the static image is tapped with a stylus); 4) to rate how closely the animation matched their feeling from X (no match at all) to 5 checkmarks (perfect match). They began with pictures resembling their own feeling states, and then (when no longer interested in exploring additional feelings) were prompted to do the other feelings in order of presentation on the tableau. The ratings
Figure 1. Sample visual representation of the MAAC tableau: the child has indicated that he or she is happy, selected the static facial image for feeling happy (circled in red), watched the animated character laughing on the screen, and endorsed 3 of 5 possible checkmarks on the rating scale (indicating that the happy animation is a moderately good match for how he or she is feeling).
and the order of feeling selection were recorded automatically by the instrument. In a separate study, clinical and nonclinical children were found to be highly accurate in identifying feeling states as portrayed by MAAC, albeit with higher accuracy at older ages [34].
State-Trait Anxiety Inventory for Children (STAIC) [35]: A 20-item self-report measure of anxiety using a 3-point scale (almost never, sometimes, always). It yields total scores for state and trait anxiety, and has been extensively normed and validated. Internal consistency in this sample: 0.833.
The Screen for Child Anxiety Related Emotional disorders (SCARED) [36]): A self-report screening tool for childhood anxiety disorders, measuring five anxiety symptom dimensions: panic/somatic, generalized anxiety, separation anxiety, social phobia and school phobia. Both parent and child versions have demonstrated good internal consistency and discriminant validity [36]. Internal consistency in this sample: 0.898.
Multidimensional Anxiety Scale for Children (MASC) [12]: A 39-item self-report measure evaluating anxious symptoms in youth aged 8 to 19 years. A total T-score can be obtained based upon the child’s gender and age. It has good internal reliability, test-retest reliability, and discriminant validity. Internal consistency in this sample: 0.881.
Children’s Depression Inventory (CDI) [37]: A 27- item self-report survey using 3-point forced choice responses, designed to measure 7 to 17 year old children’s depressive symptoms. It has been extensively validated and shows high internal consistency and moderate testretest reliability. Internal consistency in this sample: 0.780.
Koala Fear Questionnaire (KFQ) [15]: A pictorial fear scale with high convergent validity with other measures of childhood anxiety, and good internal consistency and test-retest reliability in 8 to 11 year olds. Internal consistency in this sample: 0.895.
Parents completed a standardized demographic questionnaire from the Ontario Child Health Study [26,27], to ensure comparability of clinical and control groups on key demographic factors.
2.4. Data Analysis
We estimated sample size for determining differences between clinical and control groups (discriminant validity) at about 75 per group. This sample size achieves 80% power to detect a difference of 1.1 points on the MAAC rating scale between the two group mean emotion ratings (difference of 1.0 points on the MAAC rating scale considered clinically meaningful), with estimated group standard deviations of 2.0 (based on previous findings) and a significance level (alpha) of 0.017 (adjusted from 0.05, anticipating multiple comparisons). As our analyses included nonparametric comparisons, power may be slightly lower than indicated. However, in the absence of clear methods for calculating power for nonparametric tests that do not require numerous statistical assumptions, the present estimate was used.
Using data from the children’s MAAC ratings of current feelings, exploratory factor analysis was used to extract important factors among the emotions rated. The principal factor method was used, and therefore there are no distributional assumptions on the data [38]. Power estimates advocated for such analyses vary widely. Although less than ideal, our ratio of about 10 subjects per variable, is usually considered acceptable [39], Groups were then compared on these factors using Wilcoxon rank sum tests on the distribution of mean scores on positive/fearful/calm/neutral emotions (1 for each factor). MAAC ratings do not typically follow a normal distribution, necessitating non-parametric statistics. Kurtosis and skewness vary widely by specific emotion, and are reported for the main MAAC factors below. Data were not transformed. Groups were similarly compared on number of emotions explored spontaneously. Kaplan Meier survival curves and the logrank test were used to test for differences in the “time to selection” of each emotion between the two groups of children. All emotions not spontaneously selected were considered censored for the purpose of this analysis, as some group differences in time to selection could otherwise result from group differences in the number of emotions spontaneously selected. Spearman correlations were used to examine convergent validity of current MAAC ratings (factor totals only) with the state version of the STAIC, and convergent validity of 2-week MAAC ratings with the MASC, SCARED, CDI, and Koala Fear Questionnaire. As not all questionnaires are age-normed, we examined correlations between MAAC ratings and total raw scores. Finally, MAAC factors were compared for boys and girls, and correlations between factors and age were examined.
3. RESULTS
An exploratory factor analysis yielded 4 emotion factors: positive (happy, elated, pleased, surprised) (Cronbach’s alpha = 0.83), negative (jealous, angry, irritable, disgusted) (Cronbach’s alpha = 0.76), fearful (nervous, scared) (Cronbach’s alpha = 0.71), and calm/neutral (bored, exhausted, relaxed) (Cronbach’s alpha = 0.55). Although the internal consistency for calm/neutral was questionable, internal consistencies for other factors were acceptable to good. We used a minimum eigenvalue of 1 criterion for deciding on the number of factors. The groupings of emotions in each factor suggest face validity, and these four factors explained 62% of the variance in the ratings. The remaining 3 emotions (sad, guilty, ashamed) did not map onto any factor. Given the emergence of the above factors, we predicted that children with anxiety disorders would rate themselves less positive, less calm/neutral, and more fearful than the control group (note: as our negative factor is comprised mainly of angry emotions, we did not anticipate group differences on it).
To examine discriminant validity, we compared children with anxiety disorders and controls on the MAAC factors. We used Wilcoxon rank sum tests to compare the distributions of mean MAAC ratings for each factor. Nonparametric analyses were necessary, given the kurtosis and skewness of the factors (respectively: −0.69 and −0.55 for positive; 22.41 and 4.58 for negative; 1.05 and 1.42 for fearful; 0.95 and 1.11 for calm/neutral). The mean MAAC ratings for each factor and the mean order of emotion selection for each group for all emotions are shown in Table 2, with emotions listed in the order of presentation on the MAAC tableau. On mean scores on the emotion indices identified by factor analysis, children with anxiety disorders (n = 74) rated themselves less positive (difference of −3.18; p = 0.002) and less calm/neutral (difference of −2.06; p = 0.04) than nonanxious controls (n = 83), but there were no significant differences on fearful emotions. Differences on individual emotion ratings were also explored, recognizing the limitations of such multiple comparisons. Group differences in the expected direction were found on happy, elated, pleased, surprised, nervous and jealous, but only pleased and elated remained significant after controlling for number of comparisons (Bonferroni correction).
Table 2. Mean MAAC emotion ratings and order of selection by group.
S.D. = standard deviation.
As predicted, children with anxiety disorders explored fewer emotions spontaneously than controls (score = −2.37; p = 0.02). When examining the order of emotion selection, the only significant group difference was found for “pleased”, which tended to be selected earlier in controls than in children with anxiety disorders.
To examine convergent validity, Spearman correlations between MAAC emotion ratings (factor totals only) and standardized anxiety and depression measures were determined for children ages 8 - 10 years (see Table 3; significant correlations). Ratings of current feelings were correlated with the measure of state anxiety (STAIC) and ratings of feelings for the past two weeks were correlated with the trait-focused measures (CDI, KFQ, SCARED, MASC). Significant correlations in the expected direction were found between the STAIC and negative, positive, and fearful emotions; between the CDI and negative emotions; between the SCARED and negative and fearful emotions; and between the MASC and positive and fearful emotions. After correcting p-values for the number of correlations run (i.e., p < 0.05/20 = p < 0.0 = 0.25), only the correlations between the MAAC factors and the STAIC remained significant. Analyses for the CDI, KFQ, SCARED, and MASC were re-run using current emotion ratings, in case some children interpreted items on these measures as reflecting current rather than past feelings. However, this did not result in an increase in the number of significant correlations.
To explore potential gender effects, MAAC factors were compared between boys (n = 73) and girls (n = 84). A significant difference was found on positive emotions
Table 3. Spearman correlations between MAAC factor ratings and established anxiety/depression measures.
aRatings of current feelings were correlated with the state measure, ratings of feelings in past 2 weeks were correlated with trait measures; bSTAIC: State-Trait Anxiety Inventory for Children (state version); CDI: Children’s Depression Inventory; KFQ: Koala Fear Questionnaire; SCARED: Screen for Child Anxiety-Related Emotional Disorders; MASC: Multidimensional Anxiety Scale for Children.
(difference of −2.17, p = 0.03), which were rated higher by girls. However, when gender effects were examined separately by group, this finding only remained significant for the clinical group (difference of −2.79, p = 0.007). Correlations between MAAC factors and age revealed significant negative correlations with both the positive (r = −0.283, p < 0.001) and negative (r = −0.278, p < 0.001) emotion factors, indicating that older children rated themselves lower on both of these factors. However, when correlations were examined separately by group, this finding only remained significant for the control group (r = −0.291, p = 0.008 for positive; r = −0.334, p = 0.002 for negative). Given the relative homogeneity of the sample, MAAC factors could not be compared on other demographic factors apart from parental education (comparing children of mothers and fathers with/without university degree), and no significant differences were found.
4. DISCUSSION
The MAAC factors we found were comprised of emotions of similar valence, suggesting face validity. Using these factors, we found that anxious children rated themselves lower on positive emotions and lower on calm/ neutral emotions than nonanxious children. Groups did not differ on fearful emotions, probably because our efforts to make children comfortable while completing research procedures minimized anxiety in all subjects. The low ratings on fearful emotions for both groups (Table 2) support this idea. Anxious children explored fewer emotions spontaneously (indicating greater caution) than non-anxious children. These findings are consistent with previous literature on children vulnerable to internalizing disorders. For example, behavioral inhibition (a tendency to avoid novelty and limit exploration) is a common trait in preschool children at risk for anxiety disorders longitudinally [40]. Similarly, children with anxiety disorders have been found to show high levels of negative affect generally, not limited to anxiety, with some showing low levels of positive affect as well [41,42]. Only one group difference was found in the order of emotion selection (“pleased” selected earlier by controls than clinical subjects, consistent with expectations). The additional order differences found in our previous study were not replicated, likely because analyses in the present study allowed us to examine only emotions selected spontaneously (i.e. eliminating those prompted by the research assistant).
Children’s MAAC ratings of fearful emotions showed significant correlations in the expected direction with three standardized anxiety measures (SCARED and MASC for 2-week ratings; STAIC for current ratings) and ratings of sadness showed significant correlations in the expected direction with the CDI, though only the STAIC correlations remained significant after correction for multiple comparisons. These findings support MAAC’s convergent validity with some standardized measures, particularly when used to report on current feeling states. Correspondence between MAAC ratings and the Koala Fear Questionnaire was lower, possibly due to the preponderance of items on this measure pertaining to specific fearful situations or symptoms, rather than anxious feelings. Our findings underscore the fact that MAAC is not a diagnostic measure. It is best used in the context of a thorough assessment of all aspects of children’s anxiety, including behaviors in anxiety-provoking situations, anxious cognitions, and physiological symptoms as well as subjective feelings of anxiety.
When exploring potential age and gender effects, we found lower ratings of positive and negative emotions with increasing age in the control group. This finding may reflect older, nonclinical children’s use of affect regulation strategies to defend against extreme emotions, and its absence in clinical children may indicate that this is an area of vulnerability for them. The use of such strategies increases progressively with age [43]. Older children might also have greater sensitivity to degrees of emotional expression considered socially acceptable than younger children do [43], and therefore avoid high ratings. Alternatively, younger children may have less nuanced perceptions of certain feelings than older children do [44], resulting in more extreme ratings in this group. Further studies that include measures of coping strategies, social desirability, and accuracy of emotional perception would clarify these possibilities. The gender difference showing higher ratings of positive emotions in girls than boys in the clinical group was surprising, given the association between depression and female gender in adolescents and adults [45,46]. Replication and longitudinal follow-up could determine if the presence of this positive affect in school-aged girls with anxiety disorders represents a protective factor.
Some of the above analyses were exploratory, and larger samples may be needed to validate some findings. Generalization of findings may be limited by the use of a mainly Caucasian, well-educated sample with predominantly intact families. Although groups were demographically comparable, the presence of more learning and language-related difficulties in the clinical than in the control group may have been a confounding factor in some analyses. Younger subjects may have struggled with identifying some emotions on MAAC, but it was important to include them given the potential clinical utility of MAAC in this age group. The choice of MAAC’s main character (Caucasian adolescent female) may also make it difficult for some boys and children of some ethnicities to identify with her. A new character with ethnically ambiguous features has recently been developed (see online version: https://www.sickkids.ca/MAAC/) and offered for use by children’s mental health clinicians.
Of particular relevance to clinicians, however, is the ease with which subjects indicated their feelings on MAAC. This ability speaks to the instrument’s potential to foster communication about feelings in young children. By prompting such communication, MAAC may facilitate children’s engagement with mental health providers. . In addition, by helping clinicians discuss distressing feelings with young children and teach them how to label those feelings, MAAC may prepare children to learn strategies for managing negative feelings in psychotherapy. For example, one of the first steps in cognitive behavioral therapy for anxiety is having children identify anxious feelings and situations where they occur. Although not diagnostic, MAAC’s apparent ability to distinguish anxious and nonanxious children and its correspondence with several standardized measures of childhood internalizing symptoms further support its clinical utility.
Further evaluation of MAAC in other populations and other clinical contexts is indicated. Although the present study focused on anxious children, children presenting with depressed or angry feelings may also benefit from assessment that includes MAAC, and may also show unique patterns relative to controls. Moreover, MAAC may be useful in assessing or remediating children’s ability to identify others’ emotions based on facial expression, body language, and context. Instructions would need to be adjusted, however, so that children focused on the character’s emotions rather than their own feelings. The ability to identify others’ feelings is often compromised in children on the autistic spectrum [47,48]. Similarly, applications in educational settings for children with impaired emotion recognition due to learning disabilities could be developed. MAAC may also aid in the identification or remediation of biases towards threatening or negative emotions, commonly found in children with internalizing disorders [19,20]. Attentional retraining to reduce such biases has recently been found beneficial in children with chronically high anxiety [49]. In summary, further refinement and evaluation is indicated, but by aiding communication about feelings MAAC will allow us to respond to children with more accurate empathy, potentially improving outcomes for our youngest clients.
NOTES