Paper-Based Affect Misattribution Procedure for Implicit Measurement

Abstract

The Affect Misattribution Procedure (AMP) (Payne et al., 2005) can measure implicit attitudes based on a conceptually different procedure from the Implicit Association Test (IAT) (Greenwald et al., 1998). We aim to improve the administrative advantage of the AMP by converting it into a paper-based performance test so that it can collect massive data easily without using a computer for each examinee. We conduct three Paper-based AMP experiments to achieve the typical AMP results with large effect sizes (Study 1), obtain sufficient reliability by the test-retest method (Study 2), and find a similar phenomenon of the disappearance of misattribution when participants evaluate the primes consciously prior to the target evaluation (Study 3).

Share and Cite:

Mori, K. and Uchida, A. (2015) Paper-Based Affect Misattribution Procedure for Implicit Measurement. Psychology, 6, 1531-1538. doi: 10.4236/psych.2015.612149.

1. Introduction

Researchers in implicit attitude have developed assessment procedures which are suitable for measuring attitudes without bias arising from the participants’ intentions. Such implicit measures would be useful for studying social attitudes such as racial discrimination ( Fazio, Jackson, Dunton, & Williams, 1995 ; Greenwald, McGhee, & Schwartz, 1998 ; Wittenbrink, Judd, & Park, 1997 ). Among them, the Implicit Association Test (IAT) ( Greenwald et al., 1998 ) had been most widely used because it was good for detecting implicit effects.

The advantage of non-computer-based administration was sought for the IAT by creating a variety of paper-and-pencil versions of it (e.g. Lemm, Lane, Sattler, Khan, & Nosek, 2008 ; Lowery, Hardin, & Sinclair, 2001 ; Mori, Uchida, & Imada, 2008 ; Teachman, Gapinski, Brownell, Rawlins, & Jeyaram, 2003 ). Paper-and-pencil types of implicit assessment measures can be administered in a group along with the conventional questionnaires for measuring their explicit attitudes. It would be especially preferable for social attitude research to assess collective data with both implicit and explicit measures at the same occasion ( Vargas, Sekaquaptewa, & von Hippel, 2007 ).

Payne, Cheng, Govorun, & Stewart (2005) introduced a different implicit measure, the Affect Misattribution Procedure (AMP), based on the priming effect developed originally by Murphy & Zajonc (1993) . In the AMP, an affect-laden prime followed by an ambiguous target is presented within a brief period to a participant. The participant’s task is to judge the target dichotomously as relatively pleasant or unpleasant. It has been found that, despite a warning not to be influenced by the primes, participants tend to misattribute the affection evoked by the prime to the target. Then the degree of influence of the primes on target evaluations is used to assess participants’ attitudes toward the primes.

Because the implicit measures of the AMP are dichotomous judgments rather than the reaction times, the results of the AMP are more stable than those of the IAT. It showed an average internal consistency of α = 0.88 across the six studies in Payne et al. (2005) . It also produced strong effect sizes (average d = 1.25, Payne et al., 2005 ). Thus, the AMP had become a second popular implicit measure among social cognition researchers because it was conceptually different but reliable measure of implicit social cognitions as IAT ( Blaison, Imhoff, Hühnel, Hess, & Banse, 2012 ). De Houwer, Teige-Mocigemba, Spruyt, & Moors (2009) chose these two implicit measures and reviewed them because most existing studies have focused on IAT and affective priming effects such as AMP. Bar-Anan and Nosek (2014) also treated IAT and AMP as two representative implicit measures in their review. They compared the psychometric qualities of seven indirect attitude measures; five were measuring the IAT effects in general, and the other two were measuring priming effects on which the AMP was based.

Although the AMP uses rather simple dichotomous judgments instead of the reaction times, it is still administered using personal computers. The data are only collected individually from one participant seated in front of a computer with the AMP program. However, for the social attitude research, researchers often need to obtain data from a group of participants in a classroom or lecture hall where no individual computers are available. As stated above, there have been several successful implementations of paper-and-pencil versions of the IAT. They can be administered in a similar way as questionnaires. It will be preferable if the AMP can be administered to a group of participants without using individual computers.

For this purpose, we developed a paper-based procedure of the AMP so that it could be administered in a group and termed it the “Paper-based AMP”. In the process of developing the AMP from the procedure used by Murphy & Zajonc (1993) , Payne et al. (2005) modified the presentation duration of the primes from a subliminal level (4 ms) in Murphy & Zajonc (1993) to a visible level (75 ms). We further modified the presentation duration to one second so that the presentation of the primes could be done manually without using sophisticated stimulus presentation software (or, utilizing an automatic presentation mode of a popular presentation tool such as PowerPoint). We also changed the response procedure to have the participants mark their responses on a sheet of paper instead of responding from a computer keyboard. This allowed the Paper-based AMP to be administered to a group of participants in a classroom equipped with a screen and a projector connected to a laptop computer.

In this paper, three Paper-based AMP experiments are reported. We first modified the AMP to make it suitable for group administration such that the primes and targets would be presented manually with PowerPoint slides. We also modified the response procedure so that each participant can mark his/her responses on a sheet of paper. Then we administered it to Japanese undergraduates and junior high school students in a classroom setting (Study 1). In Study 2, we administered the Paper-based AMP twice to another set of undergraduates with a two-week interval to obtain test-retest reliability data. Lastly, we examined whether the disappearance of misattribution would occur with the Paper-based AMP in the same way as with the original AMP when the participants would evaluate the primes directly before evaluating the targets (Study 3).

2. Study 1: Administration of the Paper-Based AMP to Undergraduates and Junior High School Students

2.1. Purpose

We demonstrate that the Paper-based AMP can be administered easily in a daily classroom setting in a university and a junior high school. We also examine whether the Paper-based AMP will show the same tendency toward misattribution of the primes on the evaluation of neutral stimuli immediately following them.

2.2. Method

Participants. The participants were 45 undergraduate students (27 males and 18 females; 18-26 years old) and 66 junior high school students (34 boys and 32 girls; 13-14 years old). They were all native Japanese, the former enrolled in a teacher training course at a university in Tokyo and the latter being students at a public junior high school in Nagano.

Three types of primes and neutral stimuli to be evaluated. We prepared three photos for each of the three types of primes; pleasant (babies), neutral (buildings), and unpleasant (bugs) (see Figure 1). As for the neutral stimuli to be evaluated, we chose three Tibetan characters. Payne et al. used Chinese characters for their participants in the US, but we could not use Chinese characters as “neutral” stimuli because Japanese participants are familiar with them. Therefore, we used Tibetan characters instead (see Figure 1). There were 27 combinations (3 photos × 3 primes × 3 Tibetan characters) of slides arranged in a counterbalanced order.

Evaluation of the targets. Instead of the dichotomous judgment rendered by typing keys in Payne et al. (2005) , we asked the participants to evaluate each of the neutral stimuli, either good, neutral or bad, by drawing either “O”, “△”, or “X”, respectively. These symbols are typically used for denoting a three-level judgment in Japan.

Procedure. After having received the informed-consent procedure, those who agreed to participate in the study remained seated to begin the assessment. Then the experimenters handed out the answer sheet with 27 slots to be filled in and gave the general instructions as translated below.

“I will present Tibetan characters on the screen in front of you. Your task is to evaluate each of them whether good, bad or neutral, and mark your answer in the numbered slots of the answer sheet. There are no correct or incorrect answers; we are interested in your natural impression of them. So please answer as you feel. You don’t have to try to answer in a consistent manner. Please note that prior to each Tibetan character a photo will be shown on the same screen but ignore the photos. After each Tibetan character, I will give you about three seconds to fill in your response on the answer sheet. Some photos depict insects, so if you are too sensitive to look at them, you can quit at any time. I will show you a set of examples first. Each set contains three slides: a photo, a Tibetan character, and a slide asking for your impression of the character.”

Figure 1. Slides used in the Paper-based AMP: three types of primes; Tibetan characters; and the prompt for evaluation.

Then, the experimenters presented the example set on the screen in the front of the classroom using a projector and PowerPoint software and had the participants fill in their answers in the example slot. After that, we presented the 27 sets of slides, with the timing being about one, one, and three seconds for Prime, Target, and Evaluation slides, respectively, with five-second intervals between the three-slide sets. Please note that we presented each PowerPoint slide manually with a precision of seconds rather than milliseconds as in the original AMP.

2.3. Results

No participant left the study before completing the evaluation procedure, so the data obtained from all 111 participants were used for the analyses.

Priming effects on evaluation of Tibetan characters. The ratings on the three Tibetan characters with “O”, “△”, and “X” were converted into 1, 0, −1, respectively. Figure 2 shows the average evaluation scores in terms of the primes for undergraduates and junior high school students. Although all participants evaluated the same three Tibetan characters, their evaluations varied depending on the primes that preceded them; i.e. they evaluated the Tibetan characters as pleasant when they were preceded by baby photos but unpleasant when preceded by bug photos. We found the same tendency irrespective of the age or sex of the participant. A three-way

Figure 2. Average evaluation scores of neutral stimuli proceeded by the three types of primes; pleasant (babies), neutral (buildings), and unpleasant (bugs) (upper graph for undergraduates and lower for junior high students). Vertical bars show standard errors.

mixed (2 Age × 2 Sex × 3 Prime) ANOVA revealed only the main effect of Prime was significant (F(2, 214) = 49.62, p < 0.01, Cohen’s η2 = 0.18). The multiple comparison procedures by Holm showed that the evaluation scores preceded by bugs were significantly lower than the other two primes (alpha’s = 0.0167 and 0.0250). There were no main effects of Age and Sex, nor any interactions (Fs < 2.07, ps > 0.10). We calculated the effect sizes of the Paper-based AMP (Cohen’s d) separately for the two age groups and sex of participants; 1.22 and 0.79, for female and male undergraduates, and 0.91 and 0.63 for junior high girls and boys, respectively.

3. Study 2: Test-Retest Reliability of the Paper-Based AMP

3.1. Purpose

In order to assess the reliability of the Paper-based AMP, we administer it to the same group of undergraduates twice with a two-week interval. This is the standard procedure for the test-retest reliability assessment.

3.2. Method

Participants. The participants were a different set of 103 undergraduate students (55 males and 48 females; 19-23 years old). They were all native Japanese enrolled in a teacher-training course at a university in Nagano.

The Paper-based AMP. We administered the Paper-based AMP in the same way as in Study 1, i.e. twice with a two-week interval. We used exactly the same procedure as in Study 1.

3.3. Results

All the students agreed to participate in two consecutive Paper-based AMP evaluations, and no student failed to complete both of the assessments. We obtained similar priming effects as those in Study 1; 0.128, 0.091, and −0.220 for the Pleasant, Neutral, and Unpleasant primes, respectively, in the first experiment and 0.106, -0.008, and −0.306, respectively in the second. Cohen’s d was 0.61 and 0.58, for the first and second assessments. We calculated the correlation coefficients separately for the average ratings on each of the primes for male and female participants along with those for the combined participants (Table 1). As Table 1 shows, the reliability scores of the Paper-based AMP were in the range of 0.644 and 0.788.

4. Study 3: Disappearance of the Misattribution with Conscious Responses

4.1. Purpose

Oikawa, Aarts, & Oikawa (2011) found that affect misattribution did not occur when participants were asked to evaluate the primes consciously prior to evaluating the targets in the AMP. We examine whether the disappearance of misattribution will also be observed in the paper-based administration of the AMP.

4.2. Method

Participants. These participants were a different set of 34 undergraduate students (18 males and 16 females; 19-22 years old). They were all native Japanese pursuing a teaching license at a university in Tokyo.

The Paper-based AMP with two responses. We conducted the Paper-based AMP in the same way as in Study 1 except for one modification―having participants evaluate the primes (the photos) before evaluating the targets (the Tibetan characters). Therefore, the participants responded twice to each set of the 27 prime-target combinations in the same order as those in Study 1.

After the general instructions, we presented the 27 sets of slides, with the timing being about one, three, one, and three seconds for the Prime, Prime-Evaluation, Target, and Target-Evaluation slides, respectively, with five- second intervals between the four-slide sets. All the other procedures were the same as in Study 1.

4.3. Results

No participant left the study before completing the evaluation procedure, so the data obtained from all 34 participants were analyzed. Figure 3 shows the average evaluation scores in terms of the primes. The priming effects became smaller than those found in Study 1 but did not disappear completely, as we had hypothesized. The effect sizes became smaller than those of Studies 1 and 2, but were of medium size; d = 0.383 and 0.333 for fe-

Table 1. Test-retest correlation coefficients of the Paper-based AMP.

Figure 3. Average evaluation scores of neutral stimuli preceded by the three types of primes. Vertical bars show standard errors.

male and male students, respectively. A two-way mixed (2 Sex × 3 Prime) ANOVA revealed only the main effect of Prime reached a near significant level (F(2, 64) = 3.04, p < 0.10, Cohen’s η2 = 0.03). The main effect of Sex and the interaction were not significant (Fs < 0.06, ns).

5. Discussion

Payne et al. (2005) argued the advantages of the AMP over the IAT on three points; reliability, sensitivity, and ease of use. The Paper-based AMP was derived from the AMP, and we found it retained the same levels of reliability and sensitivity. The test-retest reliability of the Paper-based AMP (0.644 < r < 0.788) was much higher than that of the IAT as reviewed in Nosek, Greenwald, & Banaji (2007) , around r = 0.55. (Payne et al. examined only the reliability of the AMP using Cronbach’s alpha and reported α = 0.88).

The sensitivity of the Paper-based AMP measured as Cohen’s d also showed an equivalent level of the AMP. Payne et al. (2005) calculated the average effect size (= d) of the AMP from their six AMP experiments as 1.25, ranging from 0.28 to 2.44, while we obtained d scores ranging from 0.58 to 1.22. According to Cohen (1988) , an effect size of d = 0.80 or greater is considered large. It is worth noting that we obtained sufficiently high d scores (0.91 and 0.63) with junior high girls and boys who were 13-14 years old. Therefore, the Paper-based AMP can be used to assess the social attitudes of junior high school students as a class.

In terms of ease of use, the Paper-based AMP was also more advantageous than the IAT and the AMP. In the present study, we administered the Paper-based AMP only five times to collect data from 351 participants in total. It took only five minutes to conduct each Paper-based AMP session. It is also easy to analyze the data. It merely requires counting the number of “Os” and “△s” in the 27 slots. It usually takes about less than half a minute to count them. Taking into consideration the validity, reliability, sensitivity, and ease of use of the implicit assessment techniques, Payne et al. (2005) wrote about the advantages of the AMP. The Paper-based AMP is even more advantageous based on its ease of use even as it retains the same levels on the other three criteria.

Blaison et al. (2012) pointed out that the AMP is not really measuring implicit attitude. De Houwer et al. (2009) warned that priming effects on which the AMP is based can be confounded with evaluative features of the stimuli and other features. However, they also casted some limitations on the validity of the IAT. They stated that it was not clear whether the differences between IAT effects and traditional (explicit) measures are due to the implicit nature of the IAT. We did not examine the validity of the Paper-based AMP in the present study. It would be desirable to conduct another study in which the same participants take both the AMP (computer-based) and the Paper-based AMP and compare the results from the two procedures. However, the basic principle behind the Paper-based AMP is the same as that of the AMP. Therefore, we deemed that it would suffice to assume the same level of validity for the Paper-based AMP and the computer-based AMP.

In the series of present experiments, we used only three Tibetan characters. However, it would have been better to use nine Tibetan characters for each combination of the three photos of three primes, or 27 different characters as in the original AMP. We need to refine further the Paper-based AMP in the course of applications to various areas of psychological research.

Instead of a validity study, we would like to report on two experimental studies in which the Paper-based AMP was used for assessing the experimental effects. Mori and Mori used the Paper-based AMP to assess the implicit effects of artificially modified facial expressions on one’s emotions, and found that participants with artificially furrowed brows rated the targets preceded by pleasant primes less favorably ( Mori & Mori, 2010 ) and rated more favorably targets preceded by negative primes when their cheeks had been raised into an artificial smile by rubber bands ( Mori & Mori, 2013 ).

6. Conclusion

The three experiments reported in the present paper showed the usefulness of the Paper-based AMP for assessing the implicit attitudes of participants much more easily than the original AMP. The findings also suggested that the Paper-based AMP retained most of the advantages of the AMP, such as high reliability and sensitivity. It was noteworthy that it could be administered in junior high schools as demonstrated in Study 1. The Paper- based AMP might provide a new research tool for educational psychologists as well as social psychologists for assessing the implicit attitudes of students in an ordinary school setting. Since the Paper-based AMP required relatively simple responses from examinees, evaluating visual figures with only three alternatives, good, bad or neutral, it could be used for examinees as young as elementary school pupils and for elderly persons.

Acknowledgements

This research was conducted while the second author was teaching at Shinonoi Nishi Junior High School. It was supported in part by a Grant-in-Aid from the Japan Society of the Promotion of Science, (KAKENHI No. 16653054) to the first author. We are grateful to the undergraduate, junior high school students and teachers who participated in this research. We would like to express our thanks to Dr. Masanori Oikawa of Doshisha University for providing useful information about the AMP. We are indebted to Rebecca Ann Marck for her help in editing the English manuscript.

Conflicts of Interest

The authors declare no conflicts of interest.

References

[1] Bar-Anan, Y., & Nosek, B. A. (2014). A Comparative Investigation of Seven Indirect Attitude Measures. Behavior Research Methods, 46, 668-688.
http://dx.doi.org/10.3758/s13428-013-0410-6
[2] Blaison, C., Imhoff, R., Hühnel, I., Hess, U., & Banse, R. (2012). The Affect Misattribution Procedure: Hot or Not? Emotion, 12, 403-412.
http://dx.doi.org/10.1037/a0026907
[3] Cohen, J. (1988). Statistical Power Analysis for the Behavioral Sciences (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum.
[4] De Houwer, J., Teige-Mocigemba, S., Spruyt, A., & Moors, A. (2009). Implicit Measures: A Normative Analysis and Review. Psychological Bulletin, 135, 347-368.
http://dx.doi.org/10.1037/a0014211
[5] Fazio, R. H., Jackson, J. R., Dunton, B. C., & Williams, C. J. (1995). Variability in Automatic Activation as an Unobtrusive Measure of Racial Attitudes: A Bona Fide Pipeline? Journal of Personality and Social Psychology, 69, 1013-1027.
http://dx.doi.org/10.1037/0022-3514.69.6.1013
[6] Greenwald, A. G., McGhee, D. E., & Schwartz, J. K. L. (1998). Measuring Individual Differences in Implicit Cognition: The Implicit Association Test. Journal of Personality and Social Psychology, 74, 1464-1480.
http://dx.doi.org/10.1037/0022-3514.74.6.1464
[7] Lemm, K. M., Lane, K. A., Sattler, D. N., Khan, S. R., & Nosek, B. A. (2008). Assessing Implicit Cognitions with a Paper-Format Implicit Association Test. In M. A. Morrison, & T. G. Morrison (Eds.), The Psychology of Modern Prejudice (pp. 121-146). New York: Nova Science Publishers.
[8] Lowery, B. S., Hardin, C. D., & Sinclair, S. (2001). Social Influence Effects on Automatic Racial Prejudice. Journal of Personality and Social Psychology, 81, 842-855.
http://dx.doi.org/10.1037/0022-3514.81.5.842
[9] Mori, H., & Mori, K. (2013). An Implicit Assessment of the Effect of Artificial Cheek Raising: When Your Face Smiles the World Looks Nicer. Perceptual and Motor Skills, 116, 466-471.
http://dx.doi.org/10.2466/24.50.PMS.116.2.466-471
[10] Mori, K., & Mori, H. (2010). Examination of the Passive Facial Feedback Hypothesis Using an Implicit Measure: With a Furrowed Brow, Neutral Objects with Pleasant Primes Look less Appealing. Perceptual and Motor Skills, 111, 785-789.
http://dx.doi.org/10.2466/02.07.24.PMS.111.6.785-789
[11] Mori, K., Uchida, A., & Imada, R. (2008). A Paper-Format Group Performance Test for Measuring the Implicit Association of Target Concepts. Behavior Research Methods, 40, 545-555.
http://dx.doi.org/10.3758/brm.40.2.546
[12] Murphy, S. T., & Zajonc, R. B. (1993). Affect, Cognition, and Awareness: Affective Priming with Optimal and Suboptimal Stimulus Exposures. Journal of Personality and Social Psychology, 64, 723-739.
http://dx.doi.org/10.1037/0022-3514.64.5.723
[13] Nosek, B. A., Greenwald, A. G., & Banaji, M. R. (2007). The Implicit Association Test at Age 7: A Methodological and Conceptual Review. In J. A. Bargh (Ed.), Automatic Processes in Social Thinking and Behavior (pp. 265-292). New York: Psychology Press.
[14] Oikawa, M., Aarts, H., & Oikawa, H. (2011). There Is a Fire Burning in My Heart: The Role of Emotional Expression in Affective Priming Effects. Cognition and Emotion, 25, 156-163.
http://dx.doi.org/10.1080/02699931003680061
[15] Payne, B. K., Cheng, C. M., Govorun, O., & Stewart, B. D. (2005). An Inkblot for Attitude: Affect Misattribution as Implicit Measurement. Journal of Personality and Social Psychology, 89, 277-293.
http://dx.doi.org/10.1037/0022-3514.89.3.277
[16] Teachman, B. A., Gapinski, K. D., Brownell, K. D., Rawlins, M., & Jeyaram, S. (2003). Demonstrations of Implicit Anti-Fat Bias: The Impact of Providing Causal Information and Evoking Empathy. Health Psychology, 22, 68-78.
http://dx.doi.org/10.1037/0278-6133.22.1.68
[17] Vargas, P. T., Sekaquaptewa, D., & von Hippel, W. (2007). Armed Only with Paper and Pencil: “Low-Tech” Measures of Implicit Attitudes. In B. Wittenbrink, & N. Schwarz (Eds.), Implicit Measures of Attitudes (pp. 123-124). New York: Guilford Press.
[18] Wittenbrink, B., Judd, C., & Park, B. (1997). Evidence for Racial Prejudice at the Implicit Level and Its Relationship with Questionnaire Measures. Journal of Personality and Social Psychology, 72, 262-274.
http://dx.doi.org/10.1037/0022-3514.72.2.262

Copyright © 2024 by authors and Scientific Research Publishing Inc.

Creative Commons License

This work and the related PDF file are licensed under a Creative Commons Attribution 4.0 International License.