1. Introduction
Indirect measurements of attitudes can take place in a variety of ways but they have in common that respondents are unaware of either what they evaluate or the effects of their evaluation. At the input level, respondents are unaware of what they evaluate, at the process level there is no introspection available and at the output level people are unaware of the outcome (Vargas, Sekaquaptewa, & von Hippel, 2007). All three levels can be used both regarding high-tech and low-tech measures of implicit or indirect attitudes.
1.1. High-Tech and Low-Tech Measure
For quite some time implicit attitudes were measured in a high-tech manner only and mostly to access stereotypes such as racist views. Still today, measurement of implicit attitudes is associated with procedures for most researchers that require computer programs and are conducted in laboratory settings. The two techniques mostly used are the IAT (see e.g. Greenwald, Poehlman, Uhlmann, & Banaji, 2009) and the affective priming (see e.g. Fazio & Olson, 2003).
Recently, more alternative ways of measuring implicit attitudes have been used, for example regarding attitudes about work such as job satisfaction (Judge, Weiss, Kammeyer-Mueller, & Hulin, 2017). The rationalization phenomenon of employees trying to find something good about their workplace despite dissatisfaction obstructs the discovery of errors and deficiencies in respondents input. Measuring implicit attitudes fits a variety of dynamic processes such as self-regulation (Werner & Milyavskaya, 2019), intergroup behavior (Kurdi et al., 2018) and well-being (Leavitt, Fong, & Greenwald, 2011).
The Implicit Association Test (IAT) (see e.g. Greenwald & Krieger, 2006; Brendl, Markman, & Messner, 2001) and the Single-category IAT (Karpinski & Steinman, 2006) both measure the relative preferences of the subject matter. Attitudes toward either group separately can be achieved by the Go no-go association task (Nosek & Banaji, 2001) and more recently with implicit normative evaluations (Yoshida et al., 2012).
Low tech measurements of implicit attitudes can also measure either relative preferences or each group separately. The former type of low tech implicit attitude measures can be exemplified with the paper format implicit association test (Vargas, Sekaquaptewa, & von Hippel, 2007; Lemm, Lane, Sattler, Khan, & Nosek, 2008). It has been proposed that such low-tech measures regard indirect attitudes and not necessarily implicit attitudes. Although it can be argued that many low-tech measures actually grasp implicit attitudes in the present study, the concept of indirect attitudes will be sued. The experiments of the present study are built upon a technique to measure indirect attitudes without any computer and with measurement for each group separately.
The development of measures using only pen and paper has been described by Vargas, Sekaquaptewa and von Hippel (2007) with some emphasis on techniques requiring measurement of reaction time. But the reaction time measurement limits the data collection to situations where the researcher is available. Luckily, some techniques do not even require a measurement of reaction time. The scoring of a neutral subject matter is one of those measurements. For example the use of scenarios of religious behavior where participants estimate described people regarding their level of religiosity reveals how religious they are themselves (Vargas, Sekaquaptewa, & von Hippel, 2007). Other ways to measure implicit attitude with neither computer nor any timing device are the go/no-go priming task (de Paula Couto & Wentura, 2017) and persuasive messages (Johnson, Petty, Briñol, & See, 2017).
Among several pilot studies conducted prior to the previous study some were conducted within the field of political psychology where they have developed methods to measure implicit attitudes directly in questionnaires (see e.g. Ksiazkiewicz & Hedrick, 2013). In a study prior to the American president election it was concluded that both explicit and implicit prejudice were significant predictors of later vote choice (Payne, Krosnick, Pasek, Lelkes, Akhtar, & Tompson, 2010). To exemplify, in one pilot study, word fragments were tested (see e.g. Schoen, Ciofalo, & Rudow, 1989). In another pilot study, participants were requested to score pictures of open office spaces regarding their attractiveness, some pictures were with no people, some with Caucasian people and some with people from another ethnical group. Although both of these techniques seemed to work another approach was chosen for the present study. The idea is to use causal attributions of described behaviors as the measurement of indirect attitudes. The fundamental attribution error means that people make causal attribution of their own faulty behavior based more on situational causes and their successful behavior of their own based more on dispositional causes. The scoring of situational causes more and individual causes less for faulty behavior should therefore indicate a positive indirect attitude toward behaviors of human beings described in scenarios.
1.2. Explicit and Implicit Attitudes
Explicit Attitudes refer to values that are available to memory and conceivable to conscious thoughts. Such direct self-report measures work well for subject matters where there is reason to assume a reasonable truthfulness, such as job satisfaction or perceived level of stress. Implicit attitudes are assumptions that are not immediately available to conscious thoughts and fit into socially sensitive subjects where the desire to make a good impression can play a big part, such as racism or controversial political views (see e.g. Charlesworth & Banaji, 2019).
Explicit Attitudes are values that are available to memory and fit all self-reporting data where there is reason to assume a reasonable truthfulness, such as job satisfaction, stress, well-being. Implied attitudes are assumptions that are not immediately available to conscious thoughts and manifest more or less automatically. They fit into socially sensitive subjects where the desire to make a good impression can play a big part, such as racism or controversial political views (see e.g. Johnson, Petty, Briñol, & See, 2017). A considerable amount of human social cognition occurs without deliberate effort and mainly outside conscious awareness. Explicit (conscious, effortful and often slow) cognitive processes can and should be distinguished from implicit (unconscious, effortless and often fast) cognitive processes. One reason for examining both explicit and implicit attitudes is the relation between attitude and behavior. There is an ambiguity about how well attitudes predict behavior (see e.g. Kraus, 1995). It is likely that explicit attitudes and implicit attitudes predict different behaviors best and there is reason to believe that the combination of both measures will enhance the overall prediction of behavior (Spence & Townsend, 2007).
Implicit attitudes and explicit attitudes are two independent processes that complement each other. A combination of explicit and implicit attitudes predicts behavior in the best way (Lindgren, Neighbors, Gasser, Ramirez, & Cvencek, 2017). High tech measures of implicit attitudes have been successful for the measurement of racial stereotypes and have a high reliability (see e.g. Greenwald, Poehlman, Uhlmann, & Banaji, 2009; Brendl, Markman, & Messner, 2001; Fazio & Olson, 2003). But there are advantages of low teach measures of implicit attitudes such as the ability to gather data out in the field. Furthermore there is a reason to assume that explicit and implicit attitudes are independent intra-individual processes. Explicit attitudes predict behavior to a certain extent. Implicit attitudes predict impulsive behavior in a better way than the prediction between explicit attitudes and general behavior (see e.g. Vargas, Sekaquaptewa, & von Hippel, 2007).
One possible reason for the rather few studies on both explicit attitudes and implicit attitudes refers to the difficulty of making direct comparisons between explicit and implicit attitudes. The measurements are fundamentally different. Explicit attitudes are built on conscious evaluations while implicit attitudes are built on indirect conscious evaluation that indirectly measure the subject matter. Also, often the social desirability has such an impact that a low correlation should be expected which makes it difficult to compare the two different measurements. Methods to measure implicit attitudes in a more standardized way will be beneficial for the comparison of explicit and implicit attitudes.
For example, people generally are able to estimate their stress levels with a high degree of honesty. At the same time, there are several areas where social desirability affects self-estimation far too much, for example estimation of performance of their own. Many opinion polls work on an explicit level, but as soon as the subject is about something sensitive, there is reason to question a unilateral use of common self-estimation. These may include topics such as immigration, extreme opinions or support for political parties on the right or left side far away. Discrimination is one such area (see e.g. Wittenbrink, Judd, & Park, 1997). In the present study possible indications about different evaluations regarding ethnicity and gender were examined.
1.3. Causal Attributions
The use of causal attributions is built upon the logic behind the self-attribution error (see e.g. Pettigrew, 1979) which states that people have a tendency to make dispositional causal attributions for good behavior or success and dispositional attributions for bad behavior or failures. This approach to measuring implicit or indirect attitudes does not even require a time measure. Implicit attitude research has almost exclusively been conducted for subject matters where social desirability is a possible problem, like for example controversial political views, prejudice and discrimination. Most studies on implicit attitudes regarding ethnicity have reached the conclusion that spontaneous judgments are more positive of whites than blacks (see e.g. Scherer & Lambert, 2012). The present study examines differences in indirect attitudes toward Swedes and foreigners.
The rationale of the study is to develop techniques of measuring indirect attitudes in questionnaires and thereby reaching a broader range of population. The basic idea behind the specific measurements of the three experiments of the present study is that more dispositional causal attributions and less situational attribution for good behavior (and more situational causal attributions and less dispositional attribution for bad behavior) will reflect an indirect positive attitude. Likewise, more dispositional causal attributions and less situational for bad behavior will indicate an indirect negative attitude. The sum of all four measurements will give an indirect measure of attitudes toward the subject matter, in this case Swedes versus foreigners. A neutral condition makes it possible to gather data about the relative effect of indirect attitudes to Swedes compared to indirect attitudes to foreigners.
2. Experiment One
In experiment one an approach using causal attributions as the conscious evaluation was tried. The fundamental attribution error means that people make causal attribution of their own faulty behavior based on situational causes and their own successful behavior more based on individual based causes (see e.g. Fiske & Taylor, 1991; Kelley, 1967). The scoring of more situational causes and less individual causes for faulty behavior should therefore indicate an indirect attitude toward the subject matter. The participants were requested to evaluate probabilities of causes regarding the same ethnical group and other ethnical groups for both men and women. Therefore gender differences were examined.
The research question addressed differences between groups of participants making attributions about Swedish men, Swedish women, foreign men, and foreign women.
2.1. Participants
The sample consisted of criminology students at a Swedish university from second and third term. There were 87 women and 73 men in the final analysis.
2.2. Material
The scenario was a real case about a convicted offender who threatens to kill a prosecutor both via email, post and telephone calls. This scenario was manipulated in such a way that four different types were created. The four different scenarios described a Swedish male, Swedish woman, foreign man and foreign woman.
The task to fulfill was identical for all participants. They were requested to estimate the likelihood of six causal explanations to the criminal behavior. That is, how likely it is that each description can explain the behavior. These estimations were reported on 100 mm Visual Analog Scales (VAS). Three of the causal explanations were dispositional and three situational. The dispositional attributions were: Personality, motive and beliefs are congruent with the behavior. The situational attributions were: Victim of circumstances (injustice), accidental action and social pressure.
Since the behavior is negative, it was suggested that more dispositional attributions will indicate a negative implicit attribute. This has been tested before in a pilot study using a within group design.
2.3. Procedure
The participants, criminology students, were informed that the task to fulfill was part of the next course on psychology. From the complete list of students the participants were randomized into four conditions. To gather sufficient data from the sufficient number of participants, students were recruited from two different levels. After data collection differences between the different course levels were examined. Also post study interviews were conducted and finally a debriefing about the study was given to all participants during class.
2.4. Result
There was a significant effect between the four different variations of scenarios (F (3, 156) = 65.86, p < .001, ηp2 = .56). The subsequent Tukey post hoc test reveals that the participants making causal attributions about a foreign man (M = −39.45) mostly differed from the other categories followed by the Swedish man (M = 15.12) category that was significantly different from the Swedish and the foreign woman category both (M = 44.22 and 39.42 respectively). However, it was only participants reading the foreign man scenario that reported higher dispositional attributions compared to situational attributions (Table 1).
There was a clear tendency of a gender effect regarding the evaluation of the scenario describing a Swedish man. Female participants reported lower score than male participants but for the one-way ANOVA this effect was not significant. However, the univariate analysis with gender with the independent variable together resulted in a significant interaction effect (F (3, 152) = 3.12, p = .028, ηp2 = .12). The interaction effect is due to the fact that data from female participants indicates a favoritism for Swedish and foreign female offenders both while there is no such equivalent effect for male participants.
![]()
Table 1. Means and standard deviations for indirect attitudes toward men (Swedish and foreign) and women (Swedish and foreign).
Note: Mean values with minus sign indicate a negative indirect attitude.
Post study interviews did not indicate any awareness of the manipulation. It seems reasonable to conclude that causal attributions can be used to measure attitudes indirectly. It is argued here that the combination of situational attributions and dispositional attributions is an adequate measurement of implicit or at least indirect attitudes. In this study participants in the foreign man condition reported negative indirect attitudes while participants in the other conditions reported positive indirect attitudes.
3. Experiment Two
In the first experiment causal attributions were used successfully to grasp indirect attitudes regarding gender and ethnicity. In that study one criminal behavior was described in a scenario and the result indicates clear discriminations regarding ethnicity and gender both. The scoring of more dispositional causal attributions and less situational causal attributions for a described bad behavior indicates a negative implicit attitude toward people described in the scenarios. However, the result may in part appear due to the special case of a criminal act evoking strong feelings.
In experiment two causal explanations were used in a similar way but with descriptions of four different work-related behavior. The participants estimated the probability of explanation in percentage from zero to hundred. Good behavior (inclusion of minorities and delegating for increased autonomy) and bad behavior (inappropriate sexual interest and unjust recruiting) both. Thus implicit attitudes regarding both ethnicity and gender were examined.
The question raised was whether the indirect attitudes were different between groups of participants reading the same scenario except text about gender and ethnicity or not. Based on result from experiment one the following hypotheses were put forward.
Hypothesis 1 There will be a difference in indirect attitudes regarding ethnicity.
Hypothesis 2 There will be a difference in indirect attitudes regarding gender.
3.1. Participants
The sample consisted of 200 participants at a Swedish civil service company. There were 108 women and 92 men in the final analysis.
3.2. Material
The scenarios were manipulated in such a way that four different types were created. The different scenarios described a Swedish male, Swedish woman, foreign man and foreign woman, additionally a fifth group of participants read a neutral scenario.
The task to fulfill was identical for all participants. They were requested to estimate the likelihood of four causal explanations to the described behavior. That is how likely in percentage it is that each description can explain the behavior. These estimations were reported on 100 mm long Visual Analog Scales (VAS). There were two good behavior descriptions (inclusion of minorities and delegating for increased autonomy) and two bad behavior descriptions (inappropriate sexual interest and unjust recruiting).
It was suggested that more dispositional attributions than situational attributions for negative behavior will indicate a negative implicit attribute. This has been tested before in a study using only one scenario describing a bad behavior (a non-published pilot study).
3.3. Procedure
The data was collected prior to consulting regarding the company’s effort to increase inclusion at work. The participants were informed that the task to fulfill was part of the next coming course on psychology. From the complete list of participants the participants were randomized into the five conditions. To gather sufficient data from sufficient number of participants, participants were recruited from two different course levels. After data collection differences between the different course levels were examined. Also post study interviews were conducted and finally a debriefing about the study were given to all participants during class.
3.4. Result
There was a significant effect between the five different variations of scenarios (F (4, 190) = 3.71, p < .006, ηp2 = .18) for the descriptions of bad behavior. The subsequent Tukey post hoc test reveal that it was the participants making causal attributions about a foreign man (M = 297.40) that was significantly different from the Swedish woman category (M = 251.85). There were no other significant differences between the four experimental conditions and the control group (Table 2). Regarding scenarios describing good work-related behavior no significant differences were found between the control condition and the experimental conditions (Table 3).
There was a clear tendency of a gender effect regarding the evaluation of the scenarios describing men. Female participants reported lower score than male participants but this effect was not significant.
In short, Hypothesis 1 was supported there was a difference in implicit attitudes regarding ethnicity. Hypothesis 2 was not supported, there was no significant difference in implicit attitudes regarding gender.
Post study interviews did not indicate any awareness of the manipulation during the data collection.
4. Experiment Three
In experiment three the following hypotheses were put forward:
H1 There will be negative indirect attitudes toward foreigners as compared to Swedes and the neutral condition.
H2 There will be a same-sex favoritism.
![]()
Table 2. Mean and standard deviations for female and male participants for the sum of the two bad behavior scenarios.
![]()
Table 3. Mean and standard deviations for female and male participants for the sum of the two good behavior scenarios.
4.1. Method
Participants and design
A total of 80 undergraduates participated in the study. The experimental manipulation consisted of five different versions of scenarios of good and bad work-related behavior. The group of participants was randomized to the five different conditions; neutral, Swedish male, Swedish female, male foreigner and female foreigner.
4.2. Material
A questionnaire with scenarios of work-related behavior was created for the purpose of measuring implicit attitudes toward Swedes and foreigners. Among the scenarios there were nine describing good behavior and nine describing bad behavior of employees. Directly below each scenario there were two 100 mm visual analog scales (VAS). One VAS about the dispositional explanation of the described behavior and one about situational explanation. The text below each scenario and above each VAS was “try to estimate the probability that this behavior can be explained by…”, “personality or other individual characteristics” and “situation, context or circumstances”, respectively.
Additionally, a single item explicit attitude measurement with the text “How often could situations at work appear where it is acceptable to hire or promote a Swedish citizen in advance of a foreigner”. This explicit measure was last in the questionnaire used as a comparison to the implicit attitude measurement.
4.3. Procedure
Undergraduate students were asked to participate at the beginning of their lecture. Nobody refused so everybody present was randomized into the five conditions. They were informed about consent and debriefing both written and orally. The questionnaire was distributed at the end of lecture.
Two weeks later a debriefing session took place during the undergraduates’ lecture. At this time they were asked about the experimental manipulation. Two participants in each condition were interviewed individually for the purpose of finding out if they guessed the purpose of the measurement and design.
4.4. Result
The means indicate a negative implicit attitude for both male and female foreigners’ conditions and a greater variance for the male foreigner group compared to the other statistical groups (Table 4).
To examine these differences, one-way ANOVAs were conducted. There was a significant effect for both the sum of good behavior scenarios (F (4, 75) = 11.87, p < .001, ηp2 = .39), for the sum of bad behavior scenarios (F (4, 75) = 10.18, p < .001, ηp2 = .35) and the implicit attitude (the sum of good behavior scenarios and bad behavior scenarios) (F (4, 75) = 11.04, p < .001, ηp2 = .37).
Participants harbor negative implicit attitudes toward foreigners but the positive implicit attitudes toward Swedes are weaker than for the neutral condition. No differences were found for the explicit attitude measurement and no gender effect existed for any analyzed variable. Thus, hypothesis 1 was supported and hypothesis 2 was not supported.
Post experiment interviews indicate that the experimental manipulation was successful.
![]()
Table 4. Means and standard deviations for the sum of causal attributions of two different scenarios across five conditions.
Note: Indirect attitudes refer to the sum of good and bad behavior values, a minus sign indicate a negative indirect attitude and a plus sign a positive attitude.
5. Discussion
Measurement of implicit attitudes is associated with procedures for most researchers that require computer programs and conducted in laboratory settings (see e.g. Greenwald, McGhee, & Schwartz, 1998). There are advantages of low teach measures of implicit attitudes such as the possibility to gather data out in the field (see e.g. Vargas, Sekaquaptewa, & Hippel, 2007).
In the present study, the concept of indirect attitudes has been used in these three experiments. Although it can be argued that these indirect measures of attitude actually grasp implicit attitudes since participants were not aware of their attitudes. However, further studies should be sued before we conclude that such low-tech measures grasp implicit attitudes or not. In the first experiment an attempt was made to use causal attributions to grasp indirect attitudes which gave data indicating different implicit attitudes both regarding ethnicity and gender. However, the scenario used in the first experiment described an emotionally loaded subject matter, a criminal act. Perhaps such a behavior is threatening and triggers emotional responses that makes it difficult to generalize across other types of behaviors.
Therefore, the second experiment made use of a measurement based on scenarios describing good and bad behaviors both at work. In this study there was only one significant difference between the levels of the independent variable. Participants making causal attributions regarding a foreign man reported a more negative implicit attitude than participant making causal attributions of scenarios describing behavior of a Swedish woman. This was the case for scenarios describing bad work-related behavior, no differences were found for scenarios describing good behavior.
It is difficult to compare explicit attitudes and implicit attitudes since the measurements are different. In a study by Wittenbrink, Judd and Park (1997) an implicit prejudice effect correlated with participants’ scores on explicit racial attitude measures. However, direct measures regarding a sensitive subject like discrimination will not result in honest answers likely. Planned studies are therefore outlined to test ways to measure explicit attitudes toward discrimination that reduce the risk of social desirability effect. It involves scenarios both for explicit and implicit attitudes. Such measurements will make it easier to compare explicit attitudes with implicit attitudes.
One possible reason for the rather few studies on both explicit attitudes and implicit attitudes refers to the difficulty of making direct comparisons between explicit and implicit attitudes. The measurements are fundamentally different. Explicit attitudes are built on conscious evaluations while implicit attitudes are built on indirect conscious evaluation that indirectly measures the subject matter. Also, often the social desirability has such an impact that a low correlation should be expected which makes it difficult to compare the two different measurements. Methods to measure implicit attitudes in a more standardized way will be beneficial for the comparison of explicit and implicit attitudes.
All three experiments show that the combination of dispositional and situational attributions can be used as an indirect measure of attitudes. Implicit attitudes were more negative toward foreigners compared to Swedes but unlike the two first experiments there was no gender effect in the third experiment.
Participants harbor negative implicit attitudes toward foreigners but data from the participants in the neutral condition gave more positive implicit attitudes than any of the conditions with more personal information. One possible reason for this is that with little information in scenarios situational attributions are triggered. But the data do not support this common sense explanation. It seems more like any personal information described in scenarios will lead to more negative implicit attitudes.
The three experiments of the present study are one step toward a questionnaire that can be used as a standardized measure of indirect or implicit attitudes. There are two major benefits of measuring implicit attitudes directly with a questionnaire. Larger number of participants can be reached more easily. The questionnaire can be used together with other measurements like for example job attitudes.
The combination of situational attributions and dispositional attributions seems to be an adequate measure of implicit attitudes. However, there is an acknowledged difficulty in operationally defining implicit attitudes. An implicit attitude can be defined as any process that people are unaware of that indicates a liking or disliking that is comparable to an explicit attitude. However, it is not clear whether a more strict operational definition should be used in this line of research. Perhaps not all indirect measures of attitudes are truly implicit.
Furthermore, the attempt to measure indirect attitude in the present study was limited by situational attributions and dispositional attributions. It is not absolutely certain whether such attributions are positive or not and therefore the resulting indirect attitude might not always correspond to simple continuum like explicit attitudes. Also the present study is based on cross sectional experiments with similar samples and many more studies with different samples are called for. One important variation of the study is to ask participants to come up with different explanations of behavior as described in scenarios and then count the number of proposed dispositional versus situational causal attributions.
Future studies should also compare this type of measurement with other similar attempts like for example the use of a word fragment completion task (Johnson & Saboe, 2011). Further studies will examine this more closely and will compare more directly bad and good causal attributions for example. Data will also be conducted out in the field examining discrimination, inclusion at work as well as other subject matters.
The results show that it is possible to measure indirect attitudes in questionnaires. The practical implication is the improved comparison of direct and indirect attitudes as well as reaching a broader range of population.