Reliability and Validity of the Modified Differential Emotions Scale ( mDES ) in a Greek Sample

Positive psychology focuses on the beneficial effects of positive variables in human health. Research on positive emotions has increased vastly over the last years due to the role that these variables play on psychological health. In order to cope with the increasing need for positive emotions measurement, Izard’s (1977) Differential Emotions Scale (DES) was modified by Fredrickson so as to include a far wider set of positive emotions and to assist scientific research. The present study examines the psychometric qualities of the mDES in a sample of 11,422 Greek adults aging from 18 to 83 years old. Results showed satisfactory reliability levels on both subscales of the test. Factor analysis revealed a three-component solution in contrast to the two-component solution as proposed by the original standardization study. The difference in the factorial structure does not inhibit positive and negative emotions subscale scoring and may be attributed to cultural elements in the Greek population. Further implications are discussed.


Introduction
Emotion is an integral part of human existence (Izard, 1971(Izard, , 1972)), which plays an important role in everyday life.The World Health Organization (1946) defines health as "a state of complete physical, mental and social well-being and not merely the absence of disease or infirmity."Research on emotions' field has increased significantly over the last years due to their effect on mental health.Since Seligman introduced the field of positive psychology, the interest of researchers has focused on the positive aspect of mental health and its role on human behavior.Positive psychology is defined as the study of the conditions and processes that contribute to the flourishing or optimal functioning of people, groups and institutions (Gable & Haidt, 2005;Seligman & Csikzentmihalyi, 2000).
Working definitions of emotions and affect are used broadly and interchangeably with the same meaning by many researchers, whilst sometimes they prefer to distinguish one definition from another.Yet, despite ongoing debate (e.g., Diener, 2000;Ekman & Davidson, 1994), consensus is emerging that emotions are but a subset of the broader class of affective phenomena.For instance, Fredrickson (2001) defines emotions as "multicomponent response tendencies that unfold over relatively short time spans" (p.218).Typically, an emotion begins with an individual's assessment of the personal meaning of some antecedent event.This appraisal process may be either conscious or unconscious and it triggers a cascade of response tendencies that manifest across loosely coupled component systems, such as subjective experience, facial expression, cognitive processing and physiological changes (Fredrickson, 2001).
Furthermore, the term "affect" refers to consciously accessible feelings as a more general concept.Although affect is present within emotions (as the component of subjective experience), it is also displayed within many other affective phenomena, including physical sensations, attitudes, moods and even affective traits.Thus, emotions are distinct from affect in multiple ways.Firstly, emotions are typically about some personally meaningful circumstance (i.e., they have an object), whereas affect is often free-floating or objectless (Jenkins & Oatley, 1996;Russell & Barrett, 1999;Ryff & Singer, 2008).Additionally, emotions are characterized as brief and implicate the multiple-component systems described above, whilst affect is often more long-lasting and may be salient only at the level of subjective experience (Ekman & Davidson, 1994;Russell & Barrett, 1999).Finally, emotions are often conceptualized as fitting into discrete categories of emotion families, like fear, anger, joy, and interest.Affect, by contrast, is frequently perceived as varying along two dimensions, either pleasantness and activation (Russell & Barrett, 1999) or positive and negative emotional activation (Tellegen, Walson, & Clark, 1999).
Moreover, to advance understanding in the area of positive emotions, Barbara Fredrickson formulated a theoretical model which is known as the broaden-and-build theory of positive emotions (Fredrickson, 1998).This theory states that certain discrete positive emotions-including joy, interest, contentment, pride, and loveshare the ability to broaden people momentary thought action repertoires and build enduring personal resources.Research evidence confirms that positive emotions produce wider visual search patterns, novel and creative thoughts and actions, more inclusive social groups, as well as more flexible goals and mindsets (Ashby & Isen, 1999;Fredrickson, Mancuso, Branigan, & Tugade, 2000;Fredrickson, Cohn, Coffey, Pek, & Finkel, 2008).Furthermore, the broadening that positive emotions produce facilitates the boost in all kinds of resources; among them the psychological ones (Tugade & Fredrickson, 2004) and correct or undo the effects of negative emotions ("The Undoing Hypothesis"; Fredrickson & Levenson, 1998;Fredrickson et al., 2000).Last but not least, positive emotions mobilize mechanisms leading to well-being (Upward Spiral; Fredrickson & Joiner, 2002).The aforementioned elements have led scientists to increase studies regarding positive emotions, traits, elements, and variables.Positive emotions in particular are currently in the scientific spotlight internationally.One of the biggest problems of scientists worldwide is the need for psychometric tools in order to measure positive emotions.
In order to cope with this need for positive emotions measurement, Izard's (1977) Differential Emotions Scale (DES) was modified by Fredrickson so as to include a far wider set of positive emotions.Thus, the modified Differential Emotions Scale (mDES) was created to be a more encompassing measure of positive emotions, than the more commonly used PANAS (Positive and Negative Affect Scale), which exclusively targets high activation positive affective states (Watson, Wiese, Vaidya, & Tellegen, 1999).Building on preliminary work of Keltner and Shiota (2003), Fredrickson supplemented the original DES with eight additional discrete positive emotions: amusement, awe, contentment, gratitude, hope, love, pride and sexual desire.These joined joy, interest and eight negative emotions plus surprise, all of which appear in the original DES.She also added an item to measure sympathy.Participants were asked to think back a period of 15 days and report on how often they had felt each of 20 different emotions.Ratings were made on a 5-point scale (0 = never, 4 = most of the time).In addition to measuring discrete emotions, Fredrickson used item analyses to create separate aggregate subscales for positive and negative emotions.The Positive Emotions subscale is a composite of 9 positive emotions (all but awe), with coefficient α = 0.79.The Negative Emotions subscale is a composite of 7 negative emotions (all but embarrassment), with coefficient α = 0.69.
Having discussed the literature we can underline specific positive emotions results that may assist our study as validity criteria.In particular, positive emotions may not only benefit recovery from depression, but also contribute to mental health flourishing (Fredrickson, Tugade, Waugh, & Larkin, 2008;Lyubomirsky, King, & Diener, 2005;Richman, Kubzansky, Maselko, Kawachi, Choo, & Bauer, 2005).Relatively little research has been done on the experience of positive emotion in traumatic and stressful situations but it is an important element in psychological resilience (Frederickson et al., 2003).For example, Fredrickson and her colleagues (2003) demonstrated that positive emotions mediate the association between resilience before a crisis and the decrease in depression symptoms.Furthermore, a recent research has shown a positive association between positive emotions and valued outcomes including life satisfaction (Ellison & Fan, 2008;Salsman, Brown, Brechting, & Carlson, 2005), optimism and sense of self-worth (Whittington & Scher, 2010) and perceived meaning in life (Martos, Thege, & Steger, 2010;Steger & Frazier, 2005).Also, a wide variety of positive feelings, states and evaluations predict positive life outcomes (Lyubomirsky et al., 2005).A growing body of literature has shown positive and negative emotion-related attitudes and states to be associated with physical health, mental health and longevity.For example, in a longitudinal study of Harvard graduates, Peterson and his colleagues (1988) found that expressed bad events could predict health outcome decades later.
Concerning age differences results from several studies suggest that negative emotions are reported less often in older than younger adults (Barrick, Hutchinson, & Deckers, 1989;Gross, Carstensen, Pasupathi, Tsai, Goetestam Skorpen, & Hsu, 1997).However, some studies did not find decreases in negative emotions across the life span.Mroczek and Kolarz (1998) claim that negative emotions were negatively correlated with age only among married men.Furthermore, another study found that negative emotions decreased from age 18 until about age 60, but did not change from age 60 to age 94 (Carstensen, Pasupathi, Mayr, & Nesselroade, 2000).Concerning positive emotions, the pattern of age-related differences is less clear.There is a study that found that older adults reported slightly higher levels of positive emotions than younger adults (Gross et al., 1997), whereas another one found an increase in positive affect with age among women (Mroczek & Kolarz, 1998).Furthermore, other studies have not found significant differences in positive affect between younger and older adults (Barrick et al., 1989;Vaux & Meddin, 1987).
The purpose of this study was to examine reliability and validity indexes for the mDES in a sample of Greek citizens.In particular, we aim to provide data regarding inter-item correlations, means, standard deviations, variances, Cronbach a and factorial structure as well as relationship with specific criteria as optimism, psychological resilience, life satisfaction in terms of criterion validity.
The present data are a subset of a larger data bank of an ongoing longitudinal study, which started in 2008, examining the effects of the economic crisis on the psychological health of Greeks in relation to several variables including positive and negative emotions.The present data were collected during the years 2008 to 2014 with the help of undergraduate psychology students, who volunteered to administer the battery of tests.The volunteers were told that the purpose of the study was to examine the effects of the economic crisis on the well-being of Greeks and they were trained on the distribution, administration and collection of the questionnaires.Each student administered the battery of tests to 15 adult individuals among their social milieu.Every year approximately 100 students participated, resulting in the collection of approximately 1500 participants.Administration was done individually and was completed in approximately 20 minutes.The data were recorded on answer sheets and scanned using the 6 th Version of Remark Office OMR.
In order to examine the criterion validity of the test, some participants also filled in other scales, which were used as criteria.

Positive and Negative Emotions
The mDES (Fredrickson et al., 2003) asks participants to recall the past 2 weeks and rate their strongest experience of each of 20 specific emotions on a 5 point Likert scale (1-Not At All to 5-Extremely).We used the Greek version of the instrument (mDES, Galanakis & Stalikas, 2012).Building on preliminary work of Keltner and Shiota (2003), Fredrickson supplemented the original DES with eight additional discrete positive emotions: amusement, awe, contentment, gratitude, hope, love, pride, and sexualdesire.These joined joy, interest, and eight negative emotions plus surprise, all of which appear in the original DES.She also added an item to measure sympathy.In addition to measuring discrete emotions, Fredrickson used item analyses to create separate aggregate subscales for positive and negative emotions.The Positive Emotions subscale is a composite of nine positive emotions (all but awe), with coefficient α = 0.79.The Negative Emotions subscale is a composite of 7 negative emotions (all but embarrassment), with coefficient α = 0.69.

Subjective Happiness
The Greek version of Subjective Happiness Scale (SHS; Lyubomirsky & Lepper, 1999;Avgoustaki, Dimitriadou, & Stalikas, 2012) was used to examine the subjectivity of persons' global happiness using four items rated on a 7-point Likert scale with higher scores reflecting greater happiness (e.g., "Some people are generally very happy.They enjoy life regardless of what is going on, getting the most out of everything.To what extent does this characterization describe you?").In our sample (N = 6976), the scale demonstrated good internal consistency (Cronbach's alpha 0.77).

Life Satisfaction
The Satisfaction With Life Scale (SWLS; Diener, Emmons, Larsen, & Griffin, 1985) examines the global assessment of a person's quality of life according to his/her chosen criteria using five items rated on a 7-point Likert scale ranging from "Strongly Disagree" to "Strongly Agree (e.g., "I am satisfied with my life").We used the Greek version of the scale (Stalikas & Lakioti, 2012), which demonstrated good internal consistency in our sample (Cronbach's alpha 0.85; N = 1803).

Inspiration
The Inspiration Scale (IS; Thrash & Elliot, 2003) measures the frequency and the intensity in which individuals feel inspired using eight items rated on a 7-point Likert scale.Four items examine the frequency of the inspiration on a scale ranging from "Never" to "Very Often" (e.g., "I experience inspiration.How often does this happen?")and four items measure the intensity of the inspiration on a scale ranging from "Not At All" to "Very Strongly" (e.g., "I am inspired to do something.How deeply or strongly in general?").A total score can be also calculated.We used the Greek version of the instrument (Avgoustaki, Dimitriadou, & Stalikas, 2012; N = 1870), which demonstrated good internal consistency (Cronbach's alpha 0.94).

Hope
The Greek version of the Hope Scale (HS;Snyder et al., 1991;Moustaki & Stalikas, 2012) was used to measure individuals' sense of successful goal-directed determination and planning of ways to meet goals using eight items rated on a 4-point Likert scale ranging from "Definitely False" to "Definitely True".The subscale of Agency intends to capture the extent to which participants feel successfully determined in meeting goals (e.g., "I energetically pursue my goals"), whereas the subscale of Pathways intends to capture the extent to which participants perceive that there are available, successful plans to meet goals (e.g., "There are lots of ways around any problem").A total score can be computed.In our sample (N = 2029), the scale demonstrated good internal consistency (Cronbach's alpha 0.86).

Optimism
The Life Orientation Test (LOT;Scheier & Carver, 1985) examines the dispositional optimism or pessimism.In other words, it measures individual's tendency to believe that he/she will experience good or bad outcomes in his/her life using eight items rated on a 5-point Likert scale ranging from "Totally Disagree" to "Totally Agree".Each subscale consists of four items.The subscale of Optimism intends to capture the extent to which participants believe that good things will happen to them (e.g., "In Uncertain times, I usually expect the best"), whereas the subscale of Pessimism intends to capture the extent to which participants expect that bad outcomes will occur in their future (e.g., "If something can go wrong for me, it will").Moreover, a total score can be computed.We used the Greek version of the instrument (Moustaki & Stalikas, 2012; N = 3210), which demonstrated mediocre internal consistency (Cronbach's alpha 0.60).

Psychological Resilience
The Greek version of the Connor-Davidson Resilience Scale (CD-RISC; Connor & Davidson, 2003;Dimitriadou & Stalikas, 2012) was used to measure individuals' stress coping ability and recovery from stress using 25 items rated on a 5-point Likert scale, with higher scores reflecting greater resilience (e.g., "Can handle unpleasant feelings").The scale consists of five factors (personal competence, tolerance, acceptance of change, control and spiritual influences), but also, a total resilience score can be computed.In our sample (N = 5920), the scale demonstrated good internal consistency (Cronbach's alpha 0.90).

Psychological Health
The Greek version of the Depression Anxiety and Stress Scale (DASS; Stalikas & Flora, 2012;Lovibond & Lovibond, 1995) was used to measure three related negative emotional states: depression (e.g., "I couldn't seem to experience any positive feeling at all"), anxiety (e.g., "I found myself in situations that made me so anxious I was most relieved when they ended") and tension/stress (e.g., "I found it difficult to relax").Participants were asked to indicate the presence of 21 symptoms "over the previous week".Each item was rated from 1 (Did Not Apply to Me At All) to 4 (Applied to Me Very Much or Most of the Time).Each of the three subscales consisted of seven items.In our sample (N = 11050), the three subscales demonstrated good internal consistency (Cronbach's alphas 0.90, 0.90 and 0.89, respectively).

Results
The statistical analysis was carried out with the use of SPSS Vol.21.

Item Analysis
We estimated variances, means, and standard deviations for all 20 items of the scale in order to examine item quality and probability of dysfunctional items or polarization.According to the methodological rule we expected variances ranging from 0.5 -1.5, indicative of a normal distribution regarding the given answers (frequency of emotions experienced was rated on an anchored 5-point scale on which 1 = Never, 5 = Most of the time).Moreover, we were expecting means ranging from 2 -4 also indicative of a normal distribution regarding the answers in the validation sample.Results shown in Table 1 indicate that most items have a normal distribution regarding the sample's answers.
Item No3 (Ashamed, humiliated, disgraced) has a relatively smaller mean (M = 1.69) and variance (V = 1.215), which may be attributed to the fact that it describes a stronger negative emotion that in general population is rare.The same principle may apply to item No15 (Repentant, guilty, blameworthy) (M = 1.67,V = 1.044).Higher scores in these negative emotions may be indicative of severe psychopathology.In conclusion, item analysis shows no problematic items in the Greek version of the scale.

Item Inter Correlations
To further examine item quality we carried out a correlational analysis between all scale items.Since the scale measures two exact opposite dimensions, namely positive emotions and negative emotions, we did two separate correlational analyses, one for the positive emotions items and one for the negative emotions items.According to the methodological rule we were expecting to find positive statistical significant correlations between the items of each subscale ranging from 0.1 -0.5.This particular strength and direction of the correlation is indicative of items that measure the same variable and are complementary to one another regarding the factor variable.Negative correlations are indicative of opposite variables measurement, while null correlations are indicative of irrelevancy to the main variable.Extremely high correlations (r > 0.6) are indicative of items that probably measure the exact same thing and therefore one of them could be omitted without losing any psychometric properties.Table 2 and Table 3 show the results of the analyses.
According to the results shown in Table 2 and Table 3 all inter item correlations in the two DESMOD subscales were positive and statistically significant at the 0.01 level, ranging from r = 0.19 to 0.6 as expected.This finding is indicative of construct validity.

Reliability
We estimated the scale reliability using the Cronbach alpha index as well as split half measures.According to the analysis, the mDES can be used as a reliable tool for the assessment of positive and negative emotions in the Greek population.Specifically the Cronbach Alpha index for the scale was a = 0.751.The split half reliability index for the same 20 mDES items was Spearman-Brown Coefficient (equal and unequal length) = 0.754.Further item analysis exploring the possibility to strengthen the scale reliability if any of the items was deleted showed that this was not possible.Hypothetical deletion of items leads to reliability decrease.In light of the item analysis results (reliability if item deleted) we decided to maintain the 20 initial scale items.

Factor Analysis
In order to examine the factorial structure of the scale we proceeded to Exploratory Factor Analysis.Based on the factorial structure of the original version of the test we expected different factor loadings for positive and  negative emotions items.Table 4 shows the results of the factor analysis.
According to the factor analysis and the Kaiser criterion (eigenvalues greater than 1) there seem to be three principal factors in the scale which explain 47.22% of the variable variance.The same conclusion can also be drawn from the scree plot and the Monte Carlo PCA for Parallel Analysis (three of the mDES factors' eigenvalues are greater than random eigenvalues).The first factor consists of 10 positive emotions items, and the two remaining factors consist of negative emotions items.The factor analysis confirms the original factorial structure of the scale, in which items load to factors of positive and negative emotions.Based on the loading table there seem to be three items with double loadings.These are items No4 (Awe, wonder, amazement), No19 (Surprised, amazed, astonished) and No20 (Sympathy, concern, compassion).Items 19 and 20 also have problematic loadings in the original version of the test as they appear to describe emotions that fall between positive and negative affectivity.Nevertheless, deletion of the above items decreases the scale reliability to 0.54, thus we have decided to keep them as part of the scale, even though they are not taken into account in the calculation of the positive and negative emotions subscales scores.The same applies also in the original version of the scale.Furthermore, even though item No4 is constructed to measure positive emotions, in the original version of the scale is not counted in the score of Positive Emotions Subscale.

Criterion Validity Analysis
In order to further examine the validity of the scale we used as criteria different positive and negative emotions variables.We hypothesized that negative emotions' total score in the DESMOD would correlate positively to Stress, Depression and Anxiety and negatively to Life Satisfaction, Psychological Resilience, Optimism, Inspiration, Hope and Subjective Happiness.The opposite direction of correlations was tested for the positive emotions subscale.Results are presented in Table 5.

Norms
In order to help mental health professionals to interpret the scores of the subscales of mDES, we calculated the normalized scores using the Stanscore4 program.In Table 6 professionals and researchers can match the raw score of the two subscales to a Sten Score ranging from 1 to 10 so as to compare the individual's score with the norm.

Discussion
This study provides empirical support for the reliability and validity of the Greek version of the mDES.Adaptation was based on data collected from 11,422 individuals residing in urban areas, using common component analysis.Results showed that the scale's items have satisfactory psychometric qualities.Mean scores ranged from 1.69 to 3.58 while variance ranged from 1.089 to 1.884 per item.Mean scores per item in relation to standard deviations and variances are indicative of normal distribution as far as participants' answers are concerned.
The items with the lowest mean and variance were No3 (Ashamed, humiliated, disgraced), No5 (Contemptuous, scornful, disdainful) and No15 (Repentant, guilty, blameworthy).One possible explanation for the means and variances in these three items may be that they represent negative emotions that are not often experienced in the population.Moreover they are seldom recognized as primary negative emotions (such as fear and anger) in typical negative emotions hierarchies.
To further examine item quality we estimated item inter-correlations expecting positive statistically significant correlations ranging from 0.1 to 0.5.The analysis showed that correlations between all items ranged from r = 0.19 to r = 0.60 as expected.This finding is indicative of construct validity.
As far as reliability is concerned, the scale has satisfactory reliability.Specifically, the Cronbach Alpha index for the scale was a = 0.751.The split half reliability index for the mDES items was Spearman-Brown Coefficient (equal and unequal length) = 0.754.
Factorial structure of the mDES was examined through exploratory factor analysis.According to the factor analysis there seem to be three principal factors in the scale which explain 47.22% of the variable variance (positive and negative emotions).The first factor (Positive Emotions) consists of positive emotions items, and the two remaining factors (Secondary Negative Emotions and Primary Negative Emotions) consist of negative emotions items.The items that load in the Positive Emotions factor are: Glad, happy, joyful/Hopeful, optimistic, encouraged/Love, closeness, trust/Content, serene, peaceful/Proud, confident, self-assured/Interested, alert, curious/Grateful, appreciative, thankful/Amused, fun loving, silly/Sexual, desiring, flirtatious.The items that load in the Secondary Negative Emotions factor are: Ashamed, humiliated, disgraced/Repentant, guilty, blameworthy/Contemptuous, scornful, disdainful/Embarrassed, self-conscious, blushing/Scared, fearful, afraid.The Primary Negative Emotions factor consists of two emotions: Angry, irritated, annoyed and Sad, downhearted, unhappy.Problematic loadings appeared in items No20 (Sympathy, concern, compassion) and No19 (Surprised, amazed, astonished), which seem to account as both positive and negative emotions depending on the stimuli, in item No7 (Disgust, distaste, revulsion), which loads simultaneously in both negative emotions factors, and in item No4 (Awe, wonder, amazement), which is excluded from the computation of the positive emotions subscale of the original scale as well.Following the original standardization of the scale we also advise Greek administrators to keep items No20 (Sympathy, concern, compassion) and No19 (Surprised, amazed and astonished) as separate emotions that are not aggregated in neither the positive nor negative emotions total scores.The authors of the original scale characterize these two emotions as "other" emotions.The same principle applies for item No4 (Awe, wonder, amazement) in our sample.As for item No7 (Disgust, distaste, revulsion) we advise to be summed in the total negative emotions score of the scale.Overall, the Positive Emotions Subscale consists of nine items and the Negative Emotions Subscale consists of eight items in the Greek version of the mDES.
If we try to examine the basic conceptual difference of the two negative emotions factors we could approach Anger and Sadness as basic primary negative emotions while the negative emotions of the other negative emotions factor could be considered as secondary negative emotions.An exception to this interpretation may apply for Fear, which could also be considered as a primary negative emotion but it loads on the secondary negative emotions factor.Nevertheless, the two separate negative emotions factors are not used as separate scores.Thus, all the above mentioned items of the two negative emotions factors are taken into account in the total estimation of the negative emotions total scores of scale takers.
The study also addressed the issue of criterion validity using as criteria specific variables based on recent bibliography.As expected, the negative emotions subscale was positively correlated to stress, anxiety and depression and negatively correlated to life satisfaction, psychological resilience, optimism, inspiration, hope and subjective happiness.On the other hand, the positive emotions subscale was negatively correlated to stress, anxiety and depression and positively correlated to life satisfaction, psychological resilience, optimism, inspiration, hope and subjective happiness.These findings are indicative of the validity of the scale in the Greek population.
Regarding the limitations of our study, we should mention that reliability indexes were not calculated using test-retest methodology but only simultaneously using the Cronbach Alpha and Split Half Indexes.Moreover, all criteria validity measures were concurrent while we could also estimate validity measures over a period of time and regarding future results.
The study provides useful insights regarding the utilization of the mDES in future studies in Greek speaking populations and it could enhance positive psychology research in all levels.

Conclusion
We strongly believe that future research regarding the validation of the mDES in the Greek population could focus in more specific negative and positive emotions measures while also exploring the cultural differences between different populations regarding emotions experiencing.In all, the mDES-Greek Version can be used as a reliable and valid psychometric tool for the measurement of positive and negative emotions in the Greek population.

Table 1 .
Means, standard deviations and variance of the 20 items of mDES in the validation sample (1 = never to 5 = most of the time).
* Every correlation is significant at the 0.01 level.
* Every correlation is significant at the 0.01 level.

Table 6 .
Norms of the positive emotion and negative emotion subscales of the mDES.