Interpersonal Rejection Anxiety Questionnaire (IRAQ): Psychometric Properties of the Greek Version

The purpose of this work was to evaluate the psychometric properties of the Interpersonal Relationship Anxiety Questionnaire (IRAQ) in Greek adults of the general population. The single factor structure of IRAQ was verified for the Greek context. We used CFA with covariates (MIMIC modeling) for the evaluation of invariance in multiple groups, regressing the latent variables and indicators onto three covariates: 1) age, 2) gender, 3) relationship status. The direct effects of the covariates on interpersonal anxiety were not statistically significant. That is the mean of interpersonal anxiety is not different at different levels of the covariates. Thus, population heterogeneity was absent in IRAQ. The direct effects of the three covariates on all indicators of interpersonal anxiety were not statistically significant. Thus, measurement invariance was supported. Additionally, reliability was adequate. Finally, correlation analysis revealed expected relations with fear of interpersonal relationships and parental rejection. IRAQ is a reliable and valid measure to use in the Greek context.

Interpersonal relationships and interpersonal/social anxiety Indeed, satisfying interpersonal relationships are related with numerous positive outcomes, including positive physical health self-esteem, and a sense of well-being and life satisfaction (Diener & Seligman, 2002;Leary, 2004;Mruk, 2006;Sedikides, Oliver, & Campbell, 1994). On the other hand, interpersonal anxiety (IA) 1 limits the ability to get involved in and continue interpersonal relationships and thus to gain the potential benefits of social support and intimacy from partners enhancing positive experiences and flourishing (Ferssizidis, Kashdan, Marquart, & Steger, 2013;Safren & Heimberg, 1999).
Interpersonal anxiety is defined as fear and avoidance of situations that may involve evaluation by others-especially significant others (Safren & Heimberg, 1999;Safren, Hollander, Hart, & Heimberg, 2001). It is described as a fear of being scrutinized by other individuals because this could potentially result in a negative evaluation and/or rejection (Rapee & Heimberg, 1997;Rohner, 2014).
Excessive interpersonal anxiety is related to interpersonal distress and with a significant diminishing of the overall life quality (Kessler et al., 1994;Safren, Heimberg, Brown, & Holle, 1997).
Research evidence generally indicates that individuals with IA have dysfunctional interpersonal and cognitive styles that impair their social performance and increase the rejection possibility (Clark & Wells, 1995;Kashdan & Roberts, 2004a;Rapee & Heimberg, 1997). Specifically, during social interactions, they tend to focus extensively on making a positive impression. However, often they perceive themselves incapable of achieving this goal because anxiety disrupts their social performance (Rapee & Heimberg, 1997;Ferssizidis et al., 2013). Making every possible effort to reduce the probability of rejection, people with IA frequently attempt to control or escape anxious thoughts and emotional states by avoiding anxiety-provoking stimuli. Therefore, during social/interpersonal interactions they adopt a "protective" behaviors like limited eye contact or conversation with others to minimize the rejection possibility (Clark & Wells, 1995). However, this rejection prevention "strategy", makes the socially anxious individual less available and responsive, thus less likely to pursue activities generating positive interpersonal outcomes Kashdan, Ferssizidis, Farmer, Adams, & McKnight, 2013). The impact of this unresponsiveness and/or unavailability can be even greater on certain age categories like adolescents (Beidel & Turner, 1998).
By closely focusing extensively on signs of negative evaluation, high-IA individuals may neglect positive aspects during their social interactions, because this 1 The term Interpersonal Anxiety is used interchangeably with the term Social Anxiety as emerged by the literature review. Psychology is more consistent with their negatively distorted self-views, worldviews (Rohner, 1975(Rohner, , 1986 or mental representations (Baldwin, 1992;Epstein, 1994;Rohner & Khaleque, 2016). These distortions and biases impair their ability to respond to social interactions, a variable that is potentially fueling positive interpersonal outcomes such as intimacy (Davis, 1982;Kashdan & Roberts, 2004b), or positive resonance (Fredrickson, 2013). Perceived interpersonal rejection (see also Rohner, 2014;Rohner, 2016) may contribute to more generalized biased cognitive estimations. For example, individuals recalling a time when they felt accepted by others gave higher estimates of room temperature than those in the same room who perceived they were rejected (Conway, Tugade, Catalino, & Fredrickson, 2013;Zhong & Leonardelli, 2008). Moreover, compared to their less anxious peers, high-IA individuals are more likely: 1) to rate their sexual episodes as less enjoyable and 2) to feel less emotionally connected to their sex partners . Generally, high-IA individuals react to potential interpersonal evaluative situations with more self-focused attention than low-IA individuals (see Hartman, 1983;Hope, Gansler, & Heimberg, 1989;Rapee & Heimberg, 1997).

The origins of Interpersonal anxiety
Focusing on the origins of interpersonal anxiety, Sullivan (1953aSullivan ( , 1953b proposed it is crucial during the human development process. He postulated that individuals attempt to diminish or eliminate interpersonal anxiety by limiting rejection by significant others or attachment figures (e.g. adult attachment; Bartholomew & Horowitz, 1991;Hazan & Shaver, 1987, 1990, 1994Simpson & Rholes, 1999). During childhood the discovery that some behaviors reduce interpersonal anxiety, while others increase it is a key learning experience. Thus, children learn to modify their behavior to decrease the possibility of experiencing interpersonal anxiety (Sullivan, 1953b) by rejection from significant others.
In other words, "our interpersonal actions are designed to invite, pull, elicit, draw, entice, or evoke restricted classes of reactions from persons with whom we interact, especially from significant others" (Kiesler, 1983: p. 198 cited by Bernier & Dozier, 2002). Interpersonal transactions that include accepting behaviors decrease or eliminate interpersonal anxiety. In contrast, transactions that incorporate rejecting behaviors may increase interpersonal anxiety (Kiesler, 1996). Eliminating interpersonal anxiety may hence become the cause of T. A. Kyriazos, A. Giotsa Psychology maintaining self-preserving patterns of interpersonal behavior (Bernier & Dozier, 2002).

The Interpersonal Relationship Anxiety Questionnaire
A new interpersonal anxiety measure is the Interpersonal Relationship Anxiety Questionnaire (Rohner, 2012). It is a unidimensional self-report instrument with nine statements describing symptoms of interpersonal anxiety and it is the focus of this work.
The unidimensional structure of IRAQ was validated in all the above studies. Specifically, the validation study in the US was carried out in a sample of 78 women (Brown et al. 2014), with a mean age of 20.17 years. IRAQ was used twice on this sample with an interval of 30 days and the test-retest reliability was adequate (r = 70, p < .01). The researcher used Principal Components Analysis with Varimax rotation and all nine items were taping a single factor explaining 50% of the variance. Factor loadings ranged from .56 to .87 (Brown et al., 2014;quoted by Naz & Kausar, 2015). The Chinese version of IRAQ (Li & Rohner, 2014) was validated in a sample of 215 adolescents with a mean age of 16 years (as reproduced by Naz & Kausar, 2015). The Urdu version of IRAQ was validated both on a clinical and a non-clinical sample of 300 female adolescents Psychology (Naz & Kausar, 2015). The IRAQ factor structure was estimated by EFA and a CFA separately on each sample (N = 150 each). First, an EFA with a Varimax rotation was carried out and all nine items were taping a single factor explaining 65% of the variance, factor loading from .47 to .76 (Naz & Kausar, 2015). CFA confirmed the EFA results, loadings from .45 to .92 (Naz & Kausar, 2015). When the mean scores of adolescents with somatoform disorders and adolescents with minor medical problems on IRAQ were compared, the two groups were found to be significantly different suggesting discriminant validity (Naz & Kausar, 2015). The convergent validity of the scale (IRAQ) was assessed using the Personality Assessment Questionnaire (PAQ; Rohner & Khaleque, 2005b;Rohner, Saavedra, & Granum, 1978b). Divergent validity was evaluated using the Warmth/ Affection, a sub-scale of the Parental Acceptance-Rejection Questionnaire (PARQ; Rohner & Khaleque, 2005a;Rohner, Saavedra, & Granum, 1978a), see Naz and Kausar (2015) for details. Summing up the IRAQ validation studies, it has been validated in collectivistic and individualistic cultural contexts (Hofstede, 2001;Triandis, 1995), and special populations like female adolescents with somatoform disorders (Naz & Kausar, 2015).
The purpose of the current study (see also Figure 1) is: 1) to validate IRAQ structure in the Greek general population; 2) to examine if age, gender or relationship status affects the IRAQ latent factor of interpersonal anxiety; 3) to examine if age, gender or relationship status affects the measurement invariance of any single item (1-9) of IRAQ; 4) to examine internal consistency reliability and convergent validity of IRAQ.

Method
Participants Inclusion criteria in the study were the following 1) gender, and 2) ongoing relationship status (in a relationship or not). The sample consisted of 148 adults with a mean age of 28.24 years (SD = 9.53). Seventy-four participants were males (50%). Seventy-four participants (50%) had a close interpersonal relationship. Among the 74 participants in a relationship 30% were dating, 15% were married and 5% were living together with their partner. The mean duration of the relationship was 73.92 months, (SD = 91.08) or 6.16 years. Most participants (70%) had a university degree, 18% finished high-school, 12% had a postgraduate degree and 1% finished junior high-school. About half of the participants (44%) were employed in the public or private sector, 26% were studying, 24% were unemployed, and 6% endorsed other choices (like freelance, retired and trainee). One hundred and forty participants had a Greek-orthodox religion and 8 participants endorsed "no religion". Materials Interpersonal Relationship Anxiety Questionnaire (IRAQ) Interpersonal anxiety was measured with the Interpersonal Relationship Anxiety Questionnaire (IRAQ) by Rohner (2012). It comprises nine items describing possible emotional states of interpersonal anxiety (e.g., feeling upset, nervous T. A. Kyriazos, A. Giotsa Psychology (almost always true), 3 (sometimes true), 2 (rarely true) and 1 (almost never true). The higher the total score the more interpersonal anxiety the respondent perceives. The possible score ranges from 9 (lowest interpersonal anxiety) to 36 (highest interpersonal anxiety). Scores that range from 1 to 15 suggest normal anxiety, from 16 to 20 mild anxiety, from 21 to 27 moderate anxiety and from 28 to 36 severe anxiety (see also Naz & Kausar, 2015).
In the Adult PARQ (Roher & Khaleque, 2005a), adult participants reflect on their childhood experiences of maternal or paternal acceptance-rejection. There are two PARQ versions: mother and father versions. Statements in the two versions are identical except referencing "mother's" behavior in the mother version versus "father's" behavior in the father version. Individuals respond on a 4-point Likert-type scale from (4) "almost always true" to (1) "almost never true." The 24 items are tapping into four dimensions: Warmth/Affection, Hostility/Aggression, Indifference/Neglect and, finally, the Undifferentiated Rejection (Rohner & Khaleque, 2005). Scores of the four acceptance-rejection dimensions are summed after reverse scoring the warmth/affection factor. The overall score is a measure of perceived acceptance-rejection ranging from 24 (maximum perceived acceptance) to 96 (maximum perceived rejection). Scores at or above 150 indicate the experience of significantly more caregiver rejection than acceptance. Scores between 140 and 149 indicate that respondents experience serious rejection but not necessarily more overall rejection than acceptance. On the other hand, scores between 60 and 120 suggest parental love. Khaleque & Rohner (2002) in a meta-analysis revealed the mean alpha coefficient, aggregated across all versions of the PARQ, at .89. Moreover, mean test-retest reliability (in a period from three weeks to seven years, Median = 15 months) was .62 (Khaleque & Rohner, 2002). Finally, factor analyses of the PARQ revealed a four-factor structure (Comunian & Gielen, 2000;Gomez & Rohner, 2011;Rohner & Chaki-Sircar, 1988;Rohner & Cournoyer, 1994;Rohner, Saavedra, & Granum, 1978a

Fear of Intimacy Scale (FIS)
The Fear of Intimacy Scale (FIS; Descutner & Thelen, 1991) is a 35-item questionnaire, designed to measure anxiety about intimate, romantic relationships. The fear of intimacy is defined as "the inhibited capacity of an individual, because of anxiety, to exchange thoughts and feelings of personal significance with another individual who is highly valued" (Descutner & Thelen, 1991: p. 219). Statements (e.g. "There are people who think that I am not an easy person to get to know.") are rated on a 5-point Likert scale, ranging from 1 (not at all characteristic of me) to 5 (extremely characteristic of me). The 15 items of the scale are reversed scored.

Demographics
Participants completed a demographics questionnaire with questions addressing: age, gender, religion, level of education, employment status, and whether the respondent was partnered or un-partnered and finally the length of their interpersonal relationship.
Procedure Participants were an ad hoc sample using network sampling. More specifically 40 postgraduate students recruited about 3 -4 adults to voluntarily participate in the study. Before test-battery completion volunteers were briefed on the nature and purpose of the study by research team members and informed consent was signed, protecting anonymity and confidentiality of the responses during the data collection, analysis and storage. The option to withdraw from the study at any time was explicitly offered to volunteers. Then they completed a paper and pencil test-battery. The required completion time was approximately 15 minutes. Nor participants neither students received any rewards.

Data Screening and distributional characteristics
One-hundred and forty-eight participants took part in the study. There were T. A. Kyriazos, A. Giotsa Psychology no missing values in the dataset because volunteers were cautioned to answer all questions during recruitment.
Prior to the CFA analysis, the data were evaluated for multivariate outliers.
Evaluation of Mahalanobis distances (Mahalanobis, 1936;Rousseeuw and Van Zomeren, 1990) for each case of the dataset (Kline, 2016;Pallant, 2016;Tabachnick & Fidell, 2013) showed there were nine multivariate outliers, p < .001. All outliers were included in subsequent analyses because they were not data entry errors thus, the exclusion was unsupported (see cases in Figure 2). The final dataset was N = 148.
The means, standard deviations, skew and kurtosis of each IRAQ item are shown in Table 1 (ps > .001). Most indicator means were around 2.19, corresponding to a scale point between "rarely true" and "sometimes true".  Table 1.

Confirmatory Factor Analysis (CFA)
The hypothesized model is overidentified, with 27 df. The ratio of cases to observed variables is 16:1 and the ratio of cases to estimated parameters is 8:1. Based on theory and prior evidence the following two models were tested.
MODEL 1 is the single factor model proposed in the literature (Brown et al., 2014;Li & Rohner, 2014;Naz & Kausar, 2015). MODEL 2 is a variation of MODEL 1 with error covariances added in item 1 with item 8 and item 2 with item 3 (see also Naz & Kausar, 2015 for a similar approach). Covariances were based on the semantic similarity of the items, item 1 (afraid) is expected to have an error covariance with item 8 (nervous) and likewise item 2 (upset) with item Psychology   CFA with covariates or MIMIC modeling (c.f. Muthen, 1989;Jöreskog & Goldberger, 1975) offers an alternative technique for evaluating invariance of indicators and latent means in multiple groups, by regressing them onto covariates denoting group membership. Additionally, MIMIC models can handle small samples, even of N = 150 (Brown, 2015: pp. 273-274).  The single factor model with error covariances was used as a baseline model because it was the optimal model in the preceding CFA. Next, the following three covariates (indicating group membership) were added to examine their effects on latent means and indicator intercepts: 1) Between age and interpersonal anxiety (1 = participants with aged ≥ 25 years, N = 62 or 42%; 0 = participants aged ≤ 24 years, N = 86 or 58%); 2) Between gender and interpersonal anxiety (1 = male participants, N = 74; 2 = female participants, N = 74) and 3) Between relationship status and interpersonal anxiety (1 = partnered participants, N = 74; 2 = un-partnered participants; N = 74). The results showed that this MIMIC model provided an acceptable fit to the data: χ 2 (49) = 70.32, SRMR = .051, RMSEA = .055 (90% CI = .020 -.081, TLI = .952, CFI = .939 (see the MIMIC model in Figure 4). Two potential sources of invariance were tested: the equivalence of factor means across groups (absence of population heterogeneity) and the equivalence of IRAQ indicator intercepts Q1-Q9 (measurement invariance). Psychology

Internal Consistency Reliability
We evaluated the reliability and validity of IRAQ using Cronbach's alpha coefficient (Cronbach, 1951) to assess the internal consistency of item responses.  Table 3 for details. There was significant moderate to strong correlations, between IRAQ and the Mother Version of Adult PARQ, ranging from r s (146) = .35, p < .001 (reversed Warmth/Affection) to r s (146) = .46, p < .001 (Indifference/Neglect). The correlations between IRAQ and the Father Version of Adult PARQ were also moderate to strong, ranging from r s (146) = .30, p < .001 (reversed Warmth/Affection) to r s (146) = .51, p < .001 (Hostility/ Aggression). The above interpretations of the magnitude of correlation adopted the criteria set by Cohen (1988). Table 3 presents all the results of the correlation analysis. Correlations of IRAQ are also presented graphically in Figure 5.

Discussion
The focus of this work was to evaluate the psychometric properties of the Inter-  Note. All correlation coefficients were significant at p < .001. FIS items allocation per factor: IFC = 1,2,4,5,9,11,12,13,15,16,20,23,24,26,28;IO = 3,6,7,8,10,14,17,18,19,21,22,25,27,29,30;PFC = 31,32,33,34,35. effects of gender, age and relationship status. 2) To evaluate internal consistency reliability (α). 3) To examine the relation of IRAQ with the construct of fear of intimacy and reflections of perceived parental rejection (both by mother and father), evidencing Convergent Validity further. Instruments used to check convergent validity was the Fear of Intimacy Scale (FIS; Descutner & Thelen. 1991) and the Parental Acceptance-Rejection Questionnaire (Adult PARQ, short form by Roher & Khaleque, 2005a). For the later, both the Mather and the father version were included to check reflected rejection of adult participants during their childhood, by both parents (see Rohner, 1975Rohner, , 2004. The 3-faced construct validation method (Kyriazos, 2018a) could not be implemented because of the sample size (see also Kyriazos et al., 2018;Kyriazos 2018aKyriazos , 2018b. The main findings suggested that: 1) the unidimensional factor structure of IRAQ is tenable in the Greek cultural context. 2) a MIMIC model was tested using the optimal IRAQ CFA model controlling for the effect of gender, age and relationship status on latent factor means and indicator intercepts and no significant effects were found. This is indicative of population heterogeneity and measurement invariance (see Muthen, 1989;Timothy Brown, 2015 andWang &Wang, 2012). 3) Internal consistency reliability was adequate. 4) Significant correlations of IRAQ with fear of intimacy (Descutner & Thelen, 1991) and perceived reflected parental rejection (Rohner, 1975(Rohner, , 2004 suggested convergent validity. 5) Description of IRAQ scores followed.

T. A. Kyriazos, A. Giotsa Psychology
Examining findings in more detail, data were first checked for data entry accuracy, and the distributions of the IRAQ observed variables were examined to see if the assumption of univariate and multivariate normality was violated. The CFA was performed to examine if the unidimensional structure of IRAQ is confirmable in this Greek Adult sample of the general population. A total of two alternative CFA models were examined. First, the single factor structure of IRAQ was tested with no error covariances in the nine observed variables. This model had an inadequate fit. Then, in line with other works found in the literature (e.g. Naz & Kausar, 2015) two error variances were added; one in item 1 (afraid) with item 8 (nervous) and a second in item 2 (upset) with item 3 (anxious). These additions are plausible on the basis of semantic similarities of the items (Brown, 2015). Beyond semantic similarity, additional theoretical support for the covariance of these errors could be found in a proposed Negative Affect model by Mehrabian (1998) as described by Crawford & Henry, 2004. In this model NA was conceptualized as a second-order factor with afraid and upset as two firstorder factors, thus as basic components of NA, these items are expected to have correlated error variances. Taking into account the goodness of fit statistics and the factor loadings the fit of this model was adequate. Evidence of construct validity was supported further because all fit measures reached the suggested levels of significance (Hair et al., 2010). Additionally, this single factor structure was also established in other cultural contexts, i.e. in a US women sample (Brown et al., 2014), in a Chinese sample of adolescents (Li & Rohner, 2014), in schoolchildren from Croatia. Finally, the Urdu version of IRAQ was also found to be unidimensional, used both on a clinical and a non-clinical sample of female adolescents (Naz & Kausar, 2015). Generally, to the best of our knowledge, the above empirical findings cannot be compared to similar results, due to a lack of empirical literature on IRAQ factor structure in the general adult population.
Regarding sample power of the model, the ratio of cases to observed variables was above the sample-to-variable acceptability threshold ranging from 5:1 -10:1 for up to N = 300 (DeVellis, 2017;Tinsley & Tinsley, 1987). Respectively, the ratio of sample to estimated parameters was equally satisfactory (Kline, 2016). Hence, sample power was adequate taking into account the high-reliability coefficients of IRAQ (DeVellis, 2017;Kyriazos, 2018b;Ullman, 2013).
Moreover, the optimal CFA model was used as a measurement model in a CFA with covariates modeling (MIMIC). The covariates of gender, age, and relationship status were tested for direct effects on the latent factor means and on indicator intercepts. The MIMIC model had an adequate fit and thus potential sources of invariance in latent means and indicator intercepts were tested (Brown, 2015). First, by adding the covariate onto the interpersonal anxiety the equivalence of factor means was evaluated across groups (population heterogeneity). A significant direct effect of either the covariate of age, gender or relationship status on the interpersonal anxiety latent factor would equal population heterogeneity. This essentially would mean that mean interpersonal anxiety is different at different levels of the covariates (Brown, 2015;Wang & Wang, 2012), however, no statistically significant effects were found. The direct effects of the covariates on social anxiety were not statistically significant. Thus, no population heterogeneity was found for interpersonal anxiety as measured by IRAQ. In practice, this means that the mean of the interpersonal anxiety latent factor is not different at different levels of the three covariates tested (age, sex, relationship status). Respondents' gender, aged or being partnered/un-partnered does not influence factor means (see Brown, 2015) and Wang and Wang (2012) for details.
Second, by MIMIC modeling, we can also investigate if these three covariates have a direct impact on the indicators of IRAQ Q1-Q9. A significant direct effect of any of these covariates on an indicator intercept would suggest that the value of the indicator differs by the values of the covariate, if the social anxiety latent factor is held constant (i.e. Differential item functioning; Muthen, 1989;Brown, 2015). However, no salient direct effects were present. This suggests there is measurement invariance between the groups, i.e. the value of Q1-Q9 does not differ by the values of any of the three covariates if the social anxiety latent factor is held constant. That is, when the factor is held constant irrespectively of the respondent's gender, age or relationship status, the intercepts of each of the nine IRAQ indicators show measurement invariance when latent factor mean does not change (Brown, 2015).
Regarding IRAQ scores, comparisons are not possible due to the absence of similar empirical findings. Specifically, a search of the existing literature revealed no general adult population norms.

Conclusion
The basic finding and strength of the present study are that the unidimensional structure of IRAQ is first validated in the general adult population. A second prominent finding is that interpersonal anxiety mean is not different at various levels of the indicators irrespectively of the gender, age and relationship status of the respondent. The same is true for indicator intercepts if a latent factor is held constant. This excludes the possibility of measurement biased when using IRAQ (Damasio & Koller, 2015). Reliability and validity results were also adequate.
However, the present study has certain limitations. The most obvious is the marginally acceptable sample size and the second is the absence of additional measures to test convergent and discriminant validity because Greek versions for various measures are often unavailable. However, sample power was not bad and the model had sufficient psychometric properties (model fit, factor loadings, and reliability coefficients) to counterbalance small sample size (DeVellis, 2017). Nev-T. A. Kyriazos, A. Giotsa Psychology ertheless, results must be replicated in a larger sample. Another problem could potentially be the use of error covariances, especially in combination with the marginally acceptable sample size and no cross-validation of the results followed (Byrne, 2012;Brown, 2015). However, the obvious content similarity of the covaried items offsets these shortcomings (Brown, 2015). Future research should replicate the current study to samples including more instruments, more age groups, and different cultures. Considering all the above findings IRAQ is a valid and reliable measure of interpersonal anxiety, with reliable psychometric properties exposing measurement invariance and absence of heterogeneity for the Greek adult population.