Assessment Center Dimensions Predict Performance-Based Bonus in Business Management Settings

This study sought to add to the literature on the validity of Assessment centers (ACs) by first examining the factorial structure emerging from observers’ dimension ratings and then examining their predictive validity using a performance criterion often unavailable to researchers—performance-based bonus payment. A series of ACs specially designed for the selection of candidates for entry-mid tier management positions in a large financial corporate (n = 180) was used as the sampling frame. For candidates who were promoted to managerial position we gathered bonus information within 6 12 months of their promotion (n = 75). The dimension ratings and factorial structure of the AC were examined to reveal a 2-factor structure pertaining to cognitive and interpersonal aspects of performance. Both the original dimensions and the two factorial grades showed moderate predictive validity using performance-based bonus as the criterion: The ‘organizational commitment’ dimension best predicted bonus payment (r = .38; p < .01) and the interpersonal factorial grade best predicted bonus (standardized b = .22 p < .01), followed by the cognitive factor, after controlling for gender and tenure. The theoretical and practical implications of the findings are briefly discussed.


Introduction
Assessment centers (AC) have earned a mixed bag of reviews and opinions as instruments of employee screening and selection.Prominent authors view them as either highly effective, valid measures of predicting future job-related performance (e.g.: Cascio, 2010;Schmidt & Hunter, 1998), or a problematic, unstable method yielding inconsistent content and construct validity indices (Bowler & Woehr, 2006;Melchers, Kleinmann & Prinz, 2010).Only few, however, will dispute the impressive criterion-related validity of ACs.Persistent evidence gathered across time and various target populations point to the ability of AC-derived ratings to predict and correlate with a broad range of job-performance indices (Thornton & Gibbons, 2009).Despite abundant evidence some questions linger regarding the validity and relevance of the AC method in HR selection contexts.This paper depicts a field study aiming at two of them.
The first issue relates to the question of construct validity in AC indices: the literature proposes that dimension ratings produced by observers in ACs do not represent assessment of separate performance-related entities but rather tend to converge into either a single factor (reflecting perhaps a halo effect), or a set of factors representing the various tasks in the AC (For a summary of this issue see Thornton & Gibbons, 2009).Can observers' impressions and judgments of participant behaviors reflect separate job-related entities?
The second issue pertains to the criteria used in AC validity studies: While the literature on this subject is quite consistent, most popular criteria measures are subjective such as job-satisfaction, supervisors' evaluations, performance appraisals by the participants and peers, etc. (Arthur, Day, McNelly, & Edens, 2006;Gaugler et al., 1987).Some studies provide additional criteria to reflect work-related performance indication such as salary and organizational hierarchy (e.g.: Bray & Grant, 1966;Jansen & Stoop, 2001).These however, remain sporadic and less available in the literature, compared with "soft measures" as mentioned above.
Using bonus payment as a criterion for job-performance rarely appears in the literature on AC's predictive validity since this information may be sensitive and unavailable to researchers.It does, however, present a potential as an easily quantifiable indicator of general job performance in for-profit organizations.The literature on pay for performance models (e.g.: Werner & Dudley, 2009) suggests a closely supervised practice, in which a clear set of performance criteria is used to systematically determine bonus payments.As such it may be an interesting criterion in predicting job related performance (Ittner, Larcker, & Rajan, 1997).
The current study examined a series of ACs designed for the selection of entry-to-mid level managers from the ranks of professional employees in a large financial organization offering investment, insurance and other related services, headquartered in Israel.The study investigated both the structure of the measures produced by the observers in the AC, seeking potential factors and dimensions underlying their judgments and then looked at the predictive validity of these factors against a seldom available criterion-performance based bonus payments.We hypothesized that 1) AC dimensions will show a consistent factor structure that is not global nor an artifact of the AC tools and, 2) AC dimensions and factors will predict managerial performance, represented by performance-based bonus.

Settings and Study Design
The study was conducted in a large financial organization providing investment services to a broad range of clients, in Israel.The organization employs about 7000 individuals, about 66% of whom are based in 180 branches nationwide.Data collection took place in a series of specially designed assessment centers used for the selection of Investments department managers.The position is considered mid-tier management and is open to inside employees only, typically investment professionals without official managerial background (though many have some training in management).

Sample
One hundred and eighty candidates took part in ACs for the above job, after applying for an internal call for candidates, and passing a basic screening process based on their resumes.Of the above sample, 75 were placed in managerial positions within 6 months following the AC.For this sample, ages ranged 29 -50 (mean age = 41.00;sd = 7.65), 60% were men and 40% were women.Tenure with the company at the time of application ranged 5 -25 years.They all had at least a Bachelor's degree and 20% had a graduate level degree or equivalent.

Instruments
Assessment centers were specially designed for this position by a team of expert Industrial-Organizational psychologists working for the organization, to reflect behaviors according to the job description provided by the organization HR division.The assessment centers included: 1) A self-presentation task in which participants planned and performed a time-limited selfpresentation; 2) A discussion group, simulating professional credit-approval dilemmas in which consensus has to be reached via discussion; 3) A group "in-basket" assignment simulating daily assignments of an investment department manager and 4) A competitive 'branch promotion event' assignment in which 2 or more sub-groups competed on designing and presenting the best promotion event for a new investment product.These tasks, though tailored for the specific job and organization are based on widely used paradigms (see for example: Bray & Grant, 1966;Thornton & Gibbons, 2009).
Data from 10 groups (total n = 75) was included in this analysis.The data was summarized across 6 dimensions rated by 2 observers (one is an HR specialist and the other an I/O psychologist from outside the organization).The two observers discussed each candidate after the AC and reached consensus as for the grade on each dimension, as well as a general recommendation regarding the candidate's fit for the position.Grades were given on a Stanine scale.The dimensions were as follows: 1) General cognitive ability: problem solving, effective information processing; 2) Work style: planned, orderly performance, paying attention to technical details while keeping awareness of the group goals; 3) Interpersonal relations: open, effective and assertive communication, collaboration and sensitivity to others; 4) Service & Sales orientation: service awareness, perceiving client-oriented service as a priority, identifying and taking opportunities to broaden the business and customer base; 5) Organizational commitment: Embracing organizational values and priorities, identification with the organization and its interests; 6) Leadership potential: initiating action, assuming responsibilities and motivating others.These dimensions fit a model of generalized dimension structure presented in a meta-analysis of a large number of studies of assessment centers (Arthur, Day, McNelly, & Edens, 2006).
Demographic data was collected via a short questionnaire.Data analysis included gender, age, and tenure.
Criterion data-bonus for managerial performance: as a criterion for managerial performance we collected data regarding the managers' performance-based bonus, at the end of the year following the AC.Bonus is calculated and paid by the HR division, ranges 0 to 30 units, each unit representing a given percentage of the employee's monthly salary.Bonus is paid once a year only in branches reaching or surpassing their business objectives and goals.Differential bonus is awarded based on profits, and performance appraisal provided by the employee's supervisor.Though there is room for personal judgment, bonus sums are monitored carefully by the corporate office and supervisors are held liable for the fairness and performance congruency of the process.Thus it is assumed that this data is a valid indication of managerial performance through the corporate lens.

Procedure
After obtaining the organization's approval for data collection, we retrieved data from the corporate files making sure no identifying markers are left in the data to allow tracing individuals, thus assuring anonymity.
The data was then analyzed using SPSS 19.0 (IBM, 2012).

Results
Before testing the hypotheses we examined the distribution of our main variables and the general associations among them.Table 1 summarizes descriptive statistics and Pearson's correlations between the study variables.
The results reveal a distribution of grades and the bonus units allowing for parametric statistical analysis (Coolican, 2010).
The analysis shows that 5 of the 6 dimension ratings in the AC correlated positively with the bonus criterion."Organizational commitment" showed the strongest association while "cognitive skills" did not associate with the criterion at all.The correlations reveal association patterns within the AC dimensions suggesting underlying factors.The correlations also indicate moderate associations between various AC dimension grades, the AC final recommendation grade and the bonus value.
We then proceeded to conduct an exploratory factor analysis (EFA) to reveal potential factors within the AC dimensions.The Varimax rotated model accounted for 68% of the total variance suggesting 2 factors.Table 2 depicts the EFA results.
The 2 factor structure matches a conceptual differentiation between cognitive/intellectual aspects and interpersonal aspects of performance in the AC.We then calculated a mean score for dimensions loading on the Cognitive factor and the dimensions loading on the Interpersonal factor.We used these factorial scores in a regression analysis, as well as tenure and gender to predict the bonus value.Table 3 summarizes the results of the analysis.
The results show that a multiple R of .36(p < .01)was accounted for by the interpersonal and cognitive factor grades.Gender and tenure did not show any added value in the model, though simple Pearson correlations were significant for 3 out of the 4 predictors.

Discussion
The current study tested two hypotheses: The first posited that AC dimension grades will reflect job criteria rather than a general impression (a single factor) or factors representing the various tasks in the AC.This hypothesis was supported by our analysis, suggesting 2 underlying factors accounting for 68% of the variance in the AC dimensions.The factors pertained to skill type (cognitive vs. interpersonal) rather than task type.These results suggest that AC observers are capable of differen-tiating between psychological and behavioral qualities via-a-vis the job description and not necessarily fall victims to the halo effect or judge each task performance as a stand-alone entity (as suggested in the literature, e.g.: Bowler & Woehr, 2009;Meriac, Hoffman, Woehr, & Fleisher, 2008).The results suggest that the differentiation between the dimensions may not be specific but skill or job content related, as demonstrated by the factors found in our analysis.
The second hypothesis posited that AC dimensions will show satisfactory predictive validity using a rarely utilized criterion: performance-based bonus.This hypothesis too was supported by the analysis: Five out of the 6 dimensions showed direct associations with the bonus sum.Interestingly, the single most predictive dimension was "organizational commitment", revealing perhaps an important aspect of the organization's culture and values, but also quite congruent with the importance of commitment, motivation, and other related concepts and measures, such as conscientiousness in predicting organizational performance (Jansen, Lievens, & Kleinmann, 2011;Witt, Burke, Barrick, & Mount, 2002).Factorial grade analysis revealed that interpersonal aspects of performance in the AC as a whole were better predictors of performance-based bonus sum than cognitive aspects.Gender and tenure did not associate with the bonus, when analyzed as a part of the whole model.
The effect sizes of the association between the AC dimensions or factorial grades with the bonus criterion were moderate.This can be partly accounted for by the fact that our sample included only candidates who successfully passed the AC, which may cause a range restriction.Based on this assumption, actual associations may be significantly higher (Sackett & Yang, 2000).
In interpreting the results one should be aware of the study limitations: A relatively small sample taken from a specific industry (finance) and a specific culture.The AC tasks and criteria (dimensions) were specially developed for the organization and the position participants applied for.Despite these limitations, our sample provided adequate statistical power and suggested trends congruent with previous evidence mentioned in the literature (Arthur, Day, McNelly, & Edens, 2006;Bray & Grant, 1966;Hofman, Melchers, Blair, Klienmann, & Ladd, 2011;Thorenton & Gibbons, 2009).As such it adds to our understanding of the potential constructs represented and assessed in ACs, as well as contribute to the body of research supporting the predictive validity of ACs as applied tools in a real world.

Table 1 .
Descriptive statistics and pearson's correlations among the study variables (n = 75).

Table 2 .
EFA analysis on the AC dimension grades.