Confirmatory Factor and Invariance Analyses of the Motivation to Control Prejudiced Reactions Scale

The present research further examines the psychometric properties of the Motivation to Control Prejudiced Reactions Scale (MCPRS). Particular attention is paid to the replicability of its factor structure and its factorial equivalence across samples of university students from Western Canada (n = 235), Eastern Canada (n = 556) and the mid-Western United States of America (n = 404). Confirmatory factor analysis and invariance analysis were carried out using the Analysis of Moment Structures (AMOS) 7.0. Results showed that while the two-factor structure of the MCPRS was replicated across samples, the original model required refinement to produce acceptable model fit (i.e., each sample had a slightly different model). Partial measurement invariance also was demonstrated for a subset of items on the MCPRS. The implications of the results, in terms of future use of the MCPRS are discussed, and limitations of the current study are outlined.


Introduction
The importance of motivation in the expression of prejudice has long been established, with an acknowledgment that feelings and beliefs in contemporary modern prejudice may be rooted in motivational biases [1].Research suggests that, during inter-group interactions, majority group members expect minority group members to view them as prejudiced and possibly evaluate them negatively.It is this expectation which generates the motivation to present a non-judgemental and non-prejudiced image [2]- [4].Contemporary social norms, in which overt expressions of prejudice are discouraged, may heighten this motivation [2].
Shelton's [4] work on inter-group processes reinforces the importance of examining this motivation as appearing less prejudiced may allow for more positive interactions between majority and minority group members.In this study, white and black individuals engaged in dyadic interaction, prior to which white individuals were either instructed to try to appear unprejudiced, or not given any instruction.While white individuals' concern about appearing prejudiced had a negative impact on their personal experiences (i.e., they were more anxious during the interaction), it also had positive consequences.When instructed not to appear prejudiced, white participants were liked more by black participants compared to those who were not given any instructions.It is possible that their anxiety was not detected by their black partner or, conceivably, their anxiety was interpreted as evidence of concern, interest and/or effort [5].The findings have important implications for interracial interactions as concerns about appearing prejudiced not only influence one's own experiences in the interaction but the experiences of one's partner.
The publication of a dual-process theory of behaviour, which proposes the involvement of motivation and opportunity as determinants of prejudice (MODE), suggests that negative attitudes can be controlled if an individual has the opportunity and motivation to do so [3] [6]- [8].According to the MODE model, there are two ways that attitudes can influence behaviour: spontaneously and deliberatively [9] [10].When individuals respond spontaneously, their attitudes, once activated, guide behaviour without the individual necessarily reflecting on them [9].In contrast, deliberative responding involves a conscious reflection on relevant attitudes and a controlling element which acts to combat the automatic response [3].Motivation and opportunity (e.g., time resources and cognitive capacity) are the two factors which determine whether a response is likely to be spontaneous or deliberative.The pursuit of research consistent with this theory led to the emergence of a novel construct: motivation to control prejudice reactions.

Motivation to Control Prejudiced Reactions
Dunton and Fazio [3] developed a scale to measure this variable: The Motivation to Control Prejudiced Reactions Scale (MCPRS).In accordance with the literature, item generation was based on three criteria: 1) Concerns about appearing prejudiced to others (e.g., In today's society it is important that one not be perceived as prejudiced in any manner); 2) Concerns about appearing prejudiced to oneself (e.g., I get angry with myself when I have a thought or feeling that might be considered prejudiced); and 3) Willingness to restrain from expressing thoughts, feelings and opinions that could offend people and lead to a dispute (e.g., If I have a prejudiced thought or feeling, I keep it to myself) [3].The final 17-item scale was found to possess good scale score reliability, a finding replicated with three further samples [3].
The convergent validity of the MCPRS was assessed through its association with scores on the Modern Racism Scale (MRS; [11]) and self-reported attitudes towards black college students.Higher MCPRS scores were associated with lower (less prejudiced) scores on the MRS.Primed attitude (as measured through facilitation patterns to words with positive or negative connotations, presented in conjunction with black and white faces), and MRS scores were moderated by MCPRS scores.That is, individuals low in the motivation to control prejudiced reactions had modern racism scores consistent with their primed attitude, whereas individuals high in the motivation to control prejudice did not evidence consistency between the two indicants of prejudice [8].As MCPRS scores increased, the disparity between self-reported attitude and primed attitude also increased.Research suggests that in situations requiring a judgement that is sufficiently race-related, motivation to control prejudiced reactions is induced, which results in individuals overcorrecting their responses on explicit measures, such as the MRS, in an effort to appear unprejudiced [7] [12].
Dunton and Fazio [3] also examined the factor structure of the MCPRS.Principal components analysis (PCA) was employed to determine the component structure of the instrument.Examination of screeplots suggested a two-component solution which, when forced, was replicated in different samples.Component comparability coefficients were assessed and corresponded closely from sample to sample.The two components are: 1) Concern with Acting Prejudiced (CAP), and 2) Restraint to Avoid Dispute (RAD).The CAP component reflects concerns about appearing prejudiced to others and to oneself, as well as the avoidance of prejudiced and offensive expressions [3].The RAD component reflects willingness to suppress one's own reactions in order to avoid possible disputes with others [3].The emergence of a second component contradicted researchers' common assumption that the motivation to control prejudice stems primarily from the desire to present a favourable image (e.g., [12]).
Scores on the CAP subscale were found to be strongly correlated with egalitarianism while scores on the RAD subscale were not [13] [14].Emotional reactions to viewing prejudiced materials were more varied in participants with higher CAP scores, who evidenced both guilt and agitation, in contrast to those scoring highly on the RAD, who experienced only agitation [13].CAP and RAD scores were both found to correlate modestly with scores on a measure of external motivation to respond without prejudice; however, as hypothesised by Plant and Devine [15], only CAP scores correlated with internal motivation to respond without prejudice.This evidence of divergent validity complements the principal components analyses which suggest that the MCPRS is characterised by two dimensions.
It should be noted that many studies have not investigated the component or factor structure of the MCPRS (e.g., [6] [16] [17].Further, among the studies that have, most were conducted using samples selected from the same university as Dunton and Fazio's [3] original research and, where specified, similarly employed PCA (e.g., [13] [14] [18]- [21].Banse and Gawronski [22], who devised a conceptually similar German instrument, chose to disregard the two-factor model due to lack of convincing empirical evidence1 .Legault, Green-Demers, Grant and Chung [23] also comment on the lack of stability of the factor structure of the MCPRS.Additionally, several issues have been identified with the scale items.Comrey and Lee (1992 as cited in [24]), recommend retaining items that have factor loadings of 0.30 or higher; however, some MCPRS items fall well below this benchmark (e.g., Item 8 has loadings of 0.18 and 0.13) [3].Furthermore, the RAD component consists almost entirely of negatively keyed items.This is problematic as components or factors of this nature may lack conceptual significance and merely denote response artefacts (e.g., [25]).
The purpose of the current study is to use confirmatory factor analysis (CFA) to determine whether the predefined factor models fit an observed set of data.This is an appropriate means of assessment given Dunton and Fazio's (1997) contention that motivation to control prejudiced reactions may be partitioned into concern and restraint dimensions.Additionally, although the scale has been used outside the cultural context in which it was initially validated (e.g., Sweden: [16]), no statistical tests of equivalence across samples (i.e., measurement invariance) have been conducted.
In CFA, a hypothesised model may be tested for fit and compared with competing models which, in turn, allow for revision and refinement of an instrument [26] [27].Specification of an unrestricted model occurs initially; each group has the same structure but corresponding parameters (e.g., factor loadings, variances, covariances, and error covariances) are allowed different estimates.A sequence of tests follows, in which the corresponding parameters in each group are restricted to be equal, allowing for identification of differences among groups [26].Ascertaining invariance is important as it can show that an instrument is valid for use with groups differing in ethnicity, age, or other individual difference variables of note.

Measures
Motivation to Control Prejudiced Reactions Scale (MCPRS; [3]).The MCPRS is a 17-item scale which measures motivation to control prejudiced reactions.It has two subscales: Concern with Appearing Prejudiced (CAP, 11 items) and Restraint to Avoid Dispute (RAD, 6 items).In the current study, responses were measured using a 5-point Likert-type scale (1 = strongly disagree, 5 = strongly agree).Higher scores represent greater motivation to control prejudiced reactions.Possible range of scores for the total scale is 17 to 85; ranges for the CAP and RAD are 11 to 55 and 6 to 30, respectively.

Data Analysis
The Analysis of Moment Structures Version 7.0 (AMOS; [31]) was used to perform all analyses.According to Byrne [32], when conducting CFA, multiple criteria must be used in assessing model fit.Thus, the following estimates of fit were employed: chi-square/df ratio (i.e., Q value see [33] [34], goodness-of-fit index (GFI), adjusted goodness-of-fit index (AGFI), comparative fit index (CFI), root mean square error of approximation (RMSEA), and the Akaike Information Criterion (AIC).For these indices, the recommended cut-offs are: Q < 3.0 [34] [35]; GFI > 0.90 [36]; AGFI > 0.80 [37]; CFI > 0.90 [38]; and RMSEA < 0.05, although values up to 0.08 are considered to represent satisfactory fit [39].Finally, the AIC was used to compare fit between competing models [40], with the model having a smaller AIC value suggesting a better representation of the data.
Assuming tests of model fit to be close to the hypothesised model for all groups, multi-group factorial invariance will be conducted.Tests for equivalence of item measurements (metric invariance) and theoretical structures (structural invariance) across American and Canadian samples 2 will then be carried out [32].The null hypothesis is tested; rejection infers non-equivalence of groups and leads to the subsequent testing of increasingly restrictive models in order to identify sources of variance [32].

Results
Cronbach's α for the MCPRS was in excess of 0.70 for all samples, which suggests acceptable scale score reliability.Alpha values, 95% confidence intervals for alpha, means, and standard deviations for total MCPRS scores are provided in Table 1.
A one-way analysis of variance revealed a statistically significant difference in MCPRS scores across the three groups, F(2, 1163) = 10.47,p < 0.001.Tukey post hoc tests revealed significant differences between the two Canadian samples (p < 0.001), and the Western Canadian and mid-Western USA samples (p < 0.001).Specifically, the Western Canadian sample evidenced a significantly stronger motivation to control prejudice reactions than the other two groups.

Confirmatory Factor Analysis of the MCPRS 3
The original two-factor model hypothesised by Dunton and Fazio [3] did not appear to have good fit for any of the samples (see Table 2).
Attempts were made to refine the model.Using estimates of regression weights, items and cross-loadings were deleted if non-significant (p > 0.05).For the Western Canada sample, Item 7 was restricted to loading on the RAD factor, and Item 8 was restricted to loading on the CAP factor, as loadings of both items on the respective alternate factor proved non-significant (p > 0.05).Items 5 and 17 were deleted as these items loaded weakly on both factors (maximal loading < 0.29).The modification indices suggested that the error covariances of Items 3 and 11 should be permitted to correlate.This change was deemed permissible as inspection of these items  [4] found that some items on a scale of homonegativity appeared to possess differential interpretations across groups of American and Canadian undergraduate students.Differences also were revealed in relation to endorsement of homonegativity, with American participants being significantly more homonegative (Morrison et al., 2009). 3For the purposes of CFA, all cases with missing data were removed.revealed that both reflect affective reactions that may ensue when one evidences prejudice against a visible minority.
For the Eastern Canadian sample, Item 7 was restricted to loading on the CAP factor due to its non-significant loading on the RAD (p > 0.05).Items 8 and 17 were deleted due to non-significant loadings on either factor (p > 0.05), and Item 5 was removed due to weak loadings on both factors (0.32, RAD; 0.21, CAP).Item 12 was removed due to modification indices suggesting that its error covariance should be permitted to correlate with several other error terms.Such output suggests item redundancy.
For the mid-Western USA sample, Item 7 was restricted to loading on the RAD factor due to its non-significant loading on the CAP (p > 0.05).Item 8 was deleted due to not loading significantly on either factor (p > 0.05).Also, due to loading weakly on both factors, Items 5 and 17 were removed (maximal loading < 0.28).Item 6 was removed due to potential item redundancy (i.e., modification indices suggest that its error term should be permitted to correlate with several other error covariances).Given similar item content, correlated error terms were permitted between Items 9 and 13.
The refined two-factor models then were tested.Inspection of Table 3 reveals that fit statistics for all samples attained conventional cut-offs for model fit (i.e., Q < 3; GFI > 0.90; CFI > 0.90; RMSEA < 0.05 with 90% CI upper bounds within the desired range).As well, AIC values for the refined models were lower.

Invariance Analysis of the MCPRS
The validity of the refined two-factor model was tested for all three samples simultaneously.A test of partial measurement invariance was conducted (i.e., only elements common to all three models were constrained to be equal).Using a macro level strategy, which requests AMOS to allow different path diagrams for each group, baseline models were tailored to fit each sample.A simultaneous multi-group analysis was conducted, where all parameters were freely estimated.Following this, analysis of a restricted model occurred.Factor covariance, factor variances, and factor loadings common to all three groups were constrained to be equal.

Discussion
Findings from the current study suggest that the MCPRS possesses satisfactory psychometric properties.Across three samples, scale score reliability coefficients were adequate (>0.70) and similar to those originally reported by Dunton and Fazio [3].To date, no researchers have examined the dimensionality of this scale using CFA nor has the measure's factorial invariance been tested.The current research addresses this omission.Although the two dimensional structure of the MCPRS posited by Dunton and Fazio [3] was retained, refinements to the model were required to ensure satisfactory fit.Partial measurement invariance was demonstrated (i.e., eleven items were found to be equivalent), although it should be highlighted that the alpha values for the invariant RAD items were low (α = 0.50 − 0.61), particularly for the Eastern Canada sample.Researchers' common assumption is that a particular level of alpha (usually 0.70) is required.A low alpha value does not necessarily denote uselessness; the RAD subscale may in fact provide meaningful coverage of the factor Restraint to Avoid Dispute [42].
It should be noted that certain items caused difficulties across all samples.Items 5 and 17, which were permitted to cross-load initially, never loaded strongly enough on either factor to merit their continued inclusion, and continued to cross-load weakly notwithstanding iterative model refinement.As their removal improved the model, it is recommended that these items be deleted.Item 8, which in Dunton and Fazio's [3] study did not load strongly on either factor, was retained in one sample (Western Canadian), and Item 7 loaded on different factors for different samples.Similarly, Items 6 and 12 were removed in specific samples, as they appeared to measure redundant content (i.e., modification indices suggested that their error covariances correlated with error terms for other items).
There are a number of limitations that warrant mention.First, the data used in the current study were all collected from university students, and previous research conducted with the MCPRS has employed this same group.Second, most participants in this study were psychology students; thus, the generalisability of these findings-even when considered vis-à-vis university students-is further called into question.It is important that reliability, dimensionality, and invariance analyses be conducted with groups other than psychology students.Considering the number of differences that arose across the three samples which, given their student status, would appear to be fairly homogenous, respondents with more diverse demographic profiles may reveal further differences in factorial structure.Third, gender differences in the dimensionality and factorial invariance of the MCPRS were not examined.One would anticipate that, if pronounced differences between male and female students existed with respect to interpretation of items on the MCPRS, the indicants of model fit would have been suboptimal across samples-a result that did not occur.However, the possibility that slightly different models exist for men and women should be investigated, as should the possibility that validation coefficients may differ between these groups.Fourth, although commonly used, some psychometrists have expressed concern about the use of Likert scales in attitudinal measurement.Hodge and Gillespie [43] note this response format may be problematic because it combines direction (agree versus disagree) with intensity (e.g., strongly versus less strongly).Future iterations of this measure may benefit through examination of the construct using different response formats (e.g., phrase completion [43] [44]).
It should be noted that, to date, when employing the MCPRS, researchers have used factor scores and permitted items to load differently on the two dimensions, as dictated by the sample employed [13] [14] [18]- [21].The current study replicated the two factor structure advocated by Dunton and Fazio (1997).However, the constellations of items comprising these factors differed.Adherence to best practice is recommended, with consistent sets of items being identified as representing either Concern with Appearing Prejudiced or Restraint to Avoid Dispute.Continuity in this regard also will prove beneficial in strengthening conclusions about the validity of the MCPRS.
Despite the aforementioned caveats, the present study provides additional strands of statistical and practical support for the use of the MCPRS as a reliable and valid measure of an individual's motivation to control prejudiced reactions.Research should continue to examine this construct as it has important implications for inter-group interactions (e.g., [4]).In an ever changing society, where populations are becoming more diverse [45], approaches to eradicate the expression and endorsement of prejudice are necessary.Researchers can ensure that they advance understanding of an individual's motivation to control prejudiced reactions by attending to the specific considerations, recommendations and avenues for future inquiry outlined in this paper.

2. 1 .
ParticipantsParticipants consisted of 1195 undergraduate university students enrolled in psychology courses in Western Canada (n = 235), Eastern Canada (n = 556) and the mid-Western United States of America (n = 404) who participated in a mass survey.The age range of participants for the Eastern Canadian and American samples was 17 to 48 years (M = 20.06,SD = 2.89) and 17 to 46 years (M = 19.60,SD = 2.42), respectively.No demographic information was collected for the Western Canadian sample.

Table 1 .
Descriptive and reliability statistics for the MCPRS.
2American and Canadian samples have been shown to constitute legitimate cross cultural comparisons.Morrison, Morrison and Franklin

Table 2 .
Model fit statistics for CFA of the original two-factor model.

Table 3 .
Model fit statistics for CFA of the refined two-factor model.

Table 4 .
Multi-group comparisons of fit statistics.

Table 5 .
Descriptive and reliability statistics for MCPRS items common to all samples.