1. Introduction
Psychological assessment instruments of the distress screener style have enormous utility for clinicians and researchers alike. While there are certainly drawbacks to any self-report modality of data collection, distress screeners offer users a quick and broad glimpse into respondents’ mental functioning in a convenient package. Often, these forms yield a total distress score, sub-scores of various categories of symptoms, and an overall clinical cutoff to aid in determinations of clinical significance for diagnostics. With such tools, scholars can easily incorporate questions about distress into their research projects and counselors/clinicians can expediently ascertain a general severity level of a client’s suffering. The distress screener is a great option for epidemiologists tracking broad trends in mental health, clinical scientists exploring symptomatology as it relates to other relevant constructs, psychotherapists conducting intake assessments, and many others. Given the wide array of uses for such tests, myriad have been developed and it is crucial that they undergo continued psychometric evaluation to maximize their reliability and validity for various populations of clients and research participants. In this special issue of the journal Psychology titled “Psychometrics and Psychological Assessment,” we are eager to contribute this manuscript on the evaluation of the Brief Symptom Inventory (BSI; Derogatis, 1975) and its use with emerging adults, specifically in the context of college and university settings.
1.1. Emerging Adulthood and Higher Education
Much evidence points to the developmental phase/stage of emerging adulthood as comprising elevated distress levels, frequent identity changes, and feelings of personal/social instability. Arnett and Taber (1994) coined the term “emerging adulthood” to refer to the recently elongated developmental trajectory in many industrialized societies during which individuals strive toward: independence regarding financial and decision-making matters, establishment of long-term committed relationships, and solidification of career identity as well as other aspects of self. This age range of approximately 18 to 29 has been described as involving the following five key characteristics: identity exploration, instability, self-focus, feeling in-between, and possibilities (Arnett, 2000; Arnett, 2006; Arnett et al., 2011). These defining factors contribute to a life phase marked by challenge for many of these individuals, as they transition away from reliance on parental/guardian resources and guidance.
Within the broader context of emerging adulthood, there are specific considerations unique to the college/university environment which may prove difficult. The burdens of academic demands, financial strain, capricious housing, cultural expectations, and the forging of one’s vocational self each have effects on this population. According to Douce and Keeling (2014), there are many pressing threats to college student mental health, including socialization of excessive substance use, sexual assault and rape, and overwhelming stress from coursework. College is often one of the most socially-outgoing periods of one’s life, and these emerging adults frequently find themselves in novel situations meeting new people, forced to rebuild social support structures. This certainly can go well for some, but others will find turbulence with the instability and exploration. For instance, Durand-Bush and colleagues (2015) found that elevated stress levels in college were related to anxiety and depression, while Soest, Luhmann, & Gerstorf (2020) found loneliness increased throughout adolescence and emerging adulthood. Further, Auerbach and colleagues (2018) found that one third (1/3) of first-year college students screened positive for at least one mental disorder in the categories of mood, anxiety, or substance use, and Handal and colleagues (2015) found that distress screeners used on college emerging adults may need to be interpreted with raised clinical cutoffs due to emerging adults’ generally higher stress levels.
Findings such as those previously described point to emerging adults in college/university settings as being a particularly vulnerable and high-stress population. With this in mind, we aim to improve psychological assessment of this group by contributing to the growing body of psychometrics research on distress screeners. Specifically, this study will focus on the BSI and its use in emerging adults in higher education settings.
1.2. Brief History of the BSI
The Brief Symptom Inventory (BSI; Derogatis, 1975) is a 53-item self-report survey that was originally designed to assess psychological symptoms and distress along 9 symptom dimensions (somatization, obsessive compulsion, interpersonal sensitivity, depression, anxiety, hostility, phobic anxiety, paranoid ideation, and psychoticism) for both psychiatric patients and non-patients alike. Each of the 53 items is presented to respondents using a 5-point Likert-style scale (ranging from “not at all” to “extremely”). The original scoring method yielded nine sub-scale scores for each of the aforementioned symptom dimensions, a total distress indicator called the “Global Severity Index,” and two other indicators of the overall intensity and variety of symptoms experienced. The BSI is accessible and relatively easy to employ, requiring only about 8 to 10 minutes to administer (Derogatis & Melisaratos, 1983). Since its development, the BSI has been used in many settings, including research and applied clinical contexts. Due to its combination of breadth and convenience, it remains a popular choice for university counseling centers to aid in their intake assessment processes.
The BSI was derived from a lineage of previous assessment measures. The earliest predecessor was the Cornell Medical Index (CMI; Weider et al., 1946), which was later edited into the Hopkins Symptom Checklist (HSL; Derogatis et al., 1974), and the Symptom Checklist-90 (SCL-90; Derogatis et al., 1973), and then a revised version of the SCL-90 (SCL-90-R, Derogatis, 1975). The SCL-90-R is a multidimensional self-report inventory of 90 items, each rated on a five-point Likert-style scale, featuring nine primary symptom dimensions designed to correspond with the original clinical subscales of the Minnesota Multiphasic Personality Inventory (MMPI; Hathaway & McKinley, 1940). When the BSI was finally developed in the mid-1970’s as a shortening of the SCL-90-R, it retained these nine MMPI-aligned symptom dimensions, hence the scoring method still used today. The BSI was later shortened even further, into an 18-item assessment measure for quick screenings of somatic, depressive, and anxious symptoms (BSI-18; Derogatis, 2001). However, the fuller 53-item BSI remains one of the more widely used formats of this lineage of instruments.
1.3. Psychometrics of the BSI
There is reasonable evidence that the BSI is a reliable and valid distress screening instrument for a number of target populations. For instance, the BSI’s introductory report found adequate internal consistency (i.e., Cronbach’s alphas), strong temporal consistency (i.e., test-retest correlations), and adequate convergent validity (i.e., correlations with relevant MMPI subscales) for both the total scores and symptom dimension scores in adult outpatients (Derogatis & Melisaratos, 1983). Similarly, Boulet and Boss (1991) found adequate psychometrics of the BSI subscales using Cronbach’s alphas and correlations with the MMPI in forensic psychiatric patients. Some evidence suggests the BSI has longitudinal construct validity (Long et al., 2007) as well as cross-ethnic construct validity (Hoe & Brekke, 2009). And specific to college/university samples, there is evidence that the symptom dimension subscales have adequate internal consistency (Hayes, 1997) and the total scores have divergent validity with the construct of resilience (Pashak et al., 2018).
Although there seems to be a general consensus that the BSI is a decent distress screening instrument for emerging adults, with accumulating evidence of its reliability and validity for this population, there is uncertainty specifically regarding its factor structure. While the original authors proposed a scoring structure with nine symptom dimension subscales (Derogatis, 1975), and supported these with internal consistency estimates, this was all likely done solely to align with the then-current design of the MMPI—thus the 9-factor BSI solution was an a priori assumption, not a data-based approach to describing the structure and ontology of distress for its users. Indeed, research on the BSI since then has yielded various structural recommendations. Boulet and Boss (1991) and Daoud and Abojedi (2010) both found evidence of a single-factor solution; Hayes (1997) and Schwannauer and Chetwynd (2007) both found evidence of a 6-factor solution; and Kellett and colleagues (2004) found evidence of an 8-factor solution, just to list a few examples of such structurally diverse results. Therefore, the factor structure of the BSI in college/university-enrolled emerging adults is of interest and will be explored here.
1.4. Clinical Decision-Making with the BSI
Although nearly every psychological instrument is better able to capture reality when more variance is described, there are many situations in which reducing spectra to categories or even dichotomies is quite useful—one such need is that of clinical caseness determinations (i.e., is someone a clinical “case” or not?). For instance, researchers may need a way to determine who is eligible for inclusion in a randomized clinical trial study of a particular treatment protocol; insurance providers may desire evidence of psychological impairment in order to reimburse for services; and clinicians may benefit from having a simplified metric with which to identify meaningful improvement or recovery in their clients. Thus, many distress screeners are assigned a “clinical cutoff,” a certain score at which a hypothetical line is drawn between those of average/normative/healthy status and those who could be described as distressed and in need of treatment. Because the BSI is a quantitative broadband measure of various forms of psychological symptomatology, it is a great candidate for being assigned clinical cutoff(s) for various populations to use for decision-making scenarios.
The earliest such recommendation is in the BSI manual (Derogatis, 1993), which argues for a clinical distinction drawn at the point of a t-score of 63 on the Global Severity Index. Using Derogatis’ adult non-patient norms, this would mean 0.7 can function as the clinical caseness cutoff (i.e., scores above 0.7 would indicate clinically significant distress). However, this recommendation is not necessarily appropriate for all populations, and certainly not for emerging adults in college/university settings, as Cochran and Hale (1985) and Handal and colleagues (2015), among others, have suggested that distress levels are higher in this population. To calculate a new marker of clinical significance based on data, Jacobson and Truax (1991) offer three formulas. Formula A assumes that healthy functioning should be outside the range of the “dysfunctional” population, setting a cut-point at two standard deviations away from the mean of a patient group in the direction of health. Formula B assumes that healthy functioning should be within the range of a “normal” population, setting a cut-point at two standard deviations away from the mean of the non-patient group in the direction of distress. Formula C assumes that an appropriate decision should take into account ranges of both the “dysfunctional” and “normal” populations, hence it places the cut-point at the average between the means of the patient and non-patient groups. These each have interpretive strengths and weaknesses regarding stringency of decision-making (Jacobson & Truax, 1991), so in this study we will illustrate each formula’s suggested cut-point and determine an appropriate clinical caseness cutoff based on the sensitivity and specificity estimates produced.
1.5. Research Questions
Much is already known about the BSI and its use in various populations. However, because there are outstanding questions about the factor structure and clinical caseness cutoff of the BSI in emerging adults in college and university settings, we intend to contribute clarifying data to the literature on these issues. Specifically, we will conduct an Exploratory Factor Analysis on the items of the BSI and calculate three different caseness cutoffs to compare their relative effectiveness at identifying current therapy clients.
2. Methods
2.1. Participants
The sample was obtained by pooling together three recent archival datasets of emerging adults in college/university settings. The sample size was n = 976, comprising: n = 315 respondents from the first dataset, described in Roskos, Handal, and Ubinger’s (2010) project on family conflict resolution; n = 410 respondents from the second dataset, described in Pashak, Handal, and Scales’ (2018) project on developmental assets; and n = 251 respondents from the third dataset, described in Booms, Vanderstelt, Tunstall, Weaver, and Pashak’s (2021) project on psychodynamic implicit personality measurement. All three of the archival samples were collected at universities in the Midwestern United States, via convenience sampling of undergraduates who received partial course credit for participating. Two of the samples were located at a private religiously affiliated university and the third was at a public university. The datasets were cleaned for missing data, adjusted to align variable coding, and finally merged together for analyses in IBM SPSS Statistics, Version 28.0.1.1(14).
Ages in the sample ranged from 18 to 29 years, with an average of M = 19.44 (SD = 1.35). Regarding gender identity, the sample consisted of 699 women (71.6%), 272 men (27.9%), and 5 transgender students (0.5%). Regarding racial and ethnic identity, the sample consisted of 774 White students (79.3%), 61 Asian American or Asian students (6.3%), 35 African American, African, or Black students (3.6%), 32 Hispanic or Latinx students (3.3%), 22 Native American, Indigenous, or Pacific Islander students (2.3%), and 52 students who chose “multiracial or other” responses (5.3%). Finally, regarding psychotherapy/counseling treatment status, the sample consisted of 106 current clients (10.9%), 233 past clients (23.9%), and 637 students who had never sought therapy services (65.3%).
2.2. Measures
Multiple assessment instruments were administered to the participants, including measures of family conflict and conflict resolution in the first dataset (Roskos et al., 2010), measures of life satisfaction and developmental resilience indicators in the second dataset (Pashak et al., 2018), and measures of personality dynamics and behavioral outcomes in the third dataset (Booms et al., 2021). Common to each dataset was inclusion of a demographics form and the BSI, which we were able to retrieve from the prior projects as archival materials.
Demographics. A brief self-report form was used to collect demographic information including age, gender identity, racial/ethnic identity, and status of psychotherapy/counseling treatment history. Specifically, regarding the last variable, participants were asked to indicate whether they were currently, previously, or never in therapy (i.e., receiving psychological treatment of a psychotherapy or counseling sort). Other demographic items were collected in some of the projects, but not in a matching format able to be used across all three archival datasets.
BSI. The Brief Symptom Inventory (BSI; Derogatis, 1983) is a 53-item self-report psychopathology assessment instrument. As described throughout this manuscript, the BSI measures a variety of forms of symptomatology and produces nine symptom dimension scales as well as three indices of total distress. The survey lists symptoms of wide-ranging kinds, and asks respondents to indicate along a 5-point Likert-style scale whether they experience the symptom “0 - not at all,” “1 - a little bit,” “2 - moderately,” “3 - quite a bit” or “4 - extremely.” One example item is “#7: Pains in the heart or chest?” and another is “#50: Feelings of worthlessness?” The Global Severity Index, an indicator of overall distress level, is calculated by simply taking the mean of all responses—hence it has a potential range of 0 to 4.
2.3. Data Analysis
The planned data analyses consisted of two endeavors. The first was an exploratory factor analysis, specifically using principal axis factoring (and a promax rotation if warranted), as well as a Monte Carlo style parallel analysis bootstrapping procedure to determine the number of factors to retain. The second was the calculation of clinical caseness cutoffs according to Jacobson and Truax’s (1991) three formulas, and a comparison of their resulting sensitivity and specificity as it relates to identifying current therapy clients within the sample.
3. Results
3.1. Exploratory Factor Analysis
Before beginning the EFA, we followed Pallant’s (2020) recommendations about ensuring the dataset’s suitability for factor analysis (i.e., sampling adequacy, sphericity, sample size, and variable interrelatedness). The Kaiser-Meyer-Olkin measure of sampling adequacy was above the suggested value of 0.6 (KMO = 0.976), the Bartlett’s Test of Sphericity was statistically significant as suggested (Chi-Square = 34510.76, df = 1378, p < 0.001), the sample size was well above the suggested minimum range of 150 to 300 (n = 976), and perusal of a correlation matrix of the 53 items of the BSI yielded many noted relationships of r = 0.3 or greater as suggested. Thus, we determined the dataset to be appropriate for undergoing factor analysis.
An EFA was conducted on the 53 items of the BSI. To determine the number of factors to retain, a scree plot analysis was conducted—indicating that a single factor occurred prior to the “elbow” of the graph. That first factor had an eigenvalue of 23.14 and explained 43.67% of the overall model variance. All subsequent eigenvalues were appreciably smaller, offering only minor incremental increases in overall variance explanatory percentage (i.e., another ten factors would only increase cumulative explained variance by about 20%), and evaluation of the loadings seemed to indicate that a single-factor solution did indeed make the most sense (i.e., all items loaded at or above the recommended value of 0.32).
To further evaluate our factor retention decision, we ran a Monte Carlo style simulation parallel analysis of the dataset with 1000 permutations using O’Connor’s (2000) syntax of the “rawpar.sps” title, which is available freely at this website (https://oconnor-psych.ok.ubc.ca/nfactors/nfactors.html). The parallel analysis, as summarized in Figure 1 (truncated to the first 15 factors for better visibility), revealed that the first 12 factors were each statistically significant as compared to the permutation-generated factors. In Figure 1 the eigenvalues are displayed along the y-axis and the hypothetical factor numbers are displayed along the x-axis (again showing only the first 15, whereas the full graph would go to 53 potential factors). Although the first 12 factors were statistically significant, the “elbow” remains a sharp turn immediately after the first factor and a 12-factor solution is even less parsimonious than the originally proposed 9-factor solution, thus we concluded that the evidence points in favor of a single-factor solution. Therefore, for the remainder of the analyses, we will refer primarily to the Global Severity Index (GSI), as it embodies the single-factor solution, and we will not drill into the other sub-scale scores.
3.2. Descriptives
The BSI’s GSI in this sample of emerging adults in college/university settings ranged from 0.00 to 3.38 (out of the potential range of 0.00 to 4.00, with higher
![]()
Figure 1. Raw eigenvalues with mean and 95th percentile simulation data.
scores indicating greater distress) and had a mean of M = 0.84 (SD = 0.69). As an assumption check, we ran an ANOVA to see if there were GSI differences based on therapy client status. Indeed, those currently in therapy (M = 1.57, SD = 0.80) had higher scores than those previously in therapy (M = 0.98, SD = 0.69), who in turn had higher scores than those never in therapy (M = 0.67, SD = 0.58). The overall ANOVA was statistically significant with a large effect size (F(2, 974) = 98.78, p < 0.001, eta-squared = 0.169), and each of the Tukey’s HSD pairwise post-hoc comparisons were statistically significant as well.
3.3. Establishing Clinical Cutoff
Jacobson and Truax (1991) proposed three equations for producing clinical caseness cutoffs, to aid in operationalizing the process of measuring clinically significant change. With these equations, new clinical cutoff scores for the BSI were calculated for college emerging adults. Formula A calls for a cut-point at two standard deviations below the mean of the current client group, which landed at just below zero. Negative scores are not possible on the BSI, so this would place all participants above the cutoff and is therefore not a useful approach. Formula B calls for a cut-point at two standard deviations above the mean of the non-client group, which landed at 1.8. This cutoff meant 140 participants (14.3%) were experiencing “clinically significant” distress, whereas 836 participants (85.7%) were below the cutoff. This cutoff score of 1.8 correctly identified 56 of the 106 current clients in the sample (sensitivity = 52.8%) and correctly identified 786 of the 870 non-clients in the sample (specificity = 90.3%). Finally, Formula C calls for a cut-point set at the mean of the client and non-client groups, balanced by their respective standard deviations, which landed at 1.2 in this sample. This cutoff meant that 264 participants (27.0%) were experiencing “clinically significant” distress, whereas 712 participants (73.0%) were below the cutoff. This cutoff score of 1.2 correctly identified 71 of the 106 current clients in the sample (sensitivity = 70.0%) and correctly identified 677 of the 870 non-clients in the sample (specificity = 77.8%). Because the results of Formula C have stronger sensitivity and would yield more inclusive screening results than Formula B, and would have a summed sensitivity and specificity value approximately matching Power and colleagues’ (2013) recommendation, we conclude that 1.2 is the best clinical caseness cutoff for the BSI’s GSI in emerging adults in college and university settings.
4. Discussion
This study pooled together three archival datasets of emerging adults to psychometrically evaluate the Brief Symptom Inventory in a large college/university sample. Our goals were to contribute data to the ongoing dialogue about the BSI’s factor structure and to propose an empirically sound distress score cut-point for identifying clinical significance.
With regard to the BSI’s factor structure, we conducted an Exploratory Factor Analysis using a Monte Carlo parallel analysis procedure, which led us to conclude that a single-factor solution is the most preferable interpretation. In other words, for emerging adults in college and university settings, the BSI’s total distress score (i.e., Global Severity Index) is likely the most psychometrically accurate and meaningful indicator to use. The other proposed subscale scores according to the BSI’s original authors may provide clinicians and researchers a glimpse at the style of a respondent’s symptom manifestations, but they are not adequately statistically reliable in this population. As Boulet and Boss (1991: p. 436) put it, “perhaps the degree, but not the precise nature, of psychopathology may be measured by the BSI.” Similarly, Hayes (1997: p. 364) states “factor analyses cast doubt on the BSI’s capacity to measure the nine proposed forms of psychopathology.” So, instead of relying on the BSI to give a clear profile of a college emerging adult’s unique symptomatology, it is likely better to use the GSI as an overall indicator of distress level. And if data beyond distress screening are needed for treatment or research purposes, the BSI should be combined with further assessment via additional instruments, clinical interview, or collateral report from other informants.
With regard to the BSI’s ability to determine the presence of clinically significant distress, we calculated all three of Jacobson and Truax’s (1991) cut-point formulas and compared them on their ability to effectively detect current psychotherapy/counseling clients in this large sample. Our findings indicated that the best clinical caseness cutoff on the BSI is a GSI score of 1.2, which had 70.0% sensitivity and 77.8% specificity. Thus, if researchers or clinicians need a BSI cutoff with which to identify emerging adults in college/university settings who are experiencing clinically significant distress and who are in need of treatment, we recommend the indicator of a GSI score of 1.2 or greater.
5. Limitations
Certainly, this project had limitations, as does any research endeavor. We will acknowledge a few key issues here. First, the BSI is a self-report measure and as such a reminder is warranted that all self-report instruments are susceptible to issues impacting reliability including social desirability bias, malingering, response sets, and others. Second, our sample does have weaknesses; although the dataset reported here is a relatively large group of emerging adults spanning the full 18 to 29 age range, they are primarily younger White women and thus are not representative of all college/university emerging adults. Third, while the BSI is broad and offers a useful distress screening methodology, it is not an assessment panacea and we do not intend to portray it this way—it should almost always be paired with a detailed clinical intake interview for most therapeutic purposes (e.g., Pashak & Heron, 2022) and for many scholarly purposes as well. Fourth, the process we used—proposing clinical cutoffs based on severity data and then evaluating those cutoffs based on client status within the same sample—may have been redundant and could have artificially inflated our sensitivity and specificity estimates. A better approach may involve collecting additional samples and testing the proposed 1.2 clinical caseness cutoff in future groups of emerging adult students. Fifth and finally, equating clinical caseness with status as a current psychotherapy/counseling client makes sense at face value, but may in fact have problems. For instance, one could easily imagine that some emerging adult college students who need psychological intervention due to their extremely high distress levels are not receiving it, perhaps because they cannot afford care, cannot find time for it, or experience other various barriers to treatment access (and this would throw off sensitivity estimates). Conversely, some students who may be in only very mild distress or are experiencing no symptomatology at all perhaps seek treatment (and this would throw off specificity estimates). Therefore, additional studies could benefit the literature by testing this 1.2 clinical caseness cutoff score against additional metrics beyond client status, and we encourage future researchers to investigate these questions.
6. Conclusion
The Brief Symptom Inventory is a well-crafted psychological assessment instrument, useful for a variety of populations and purposes regarding mental health measurement. In emerging adults in college and university settings, our data provide evidence of a single-factor solution and a clinical caseness cutoff of 1.2 on the Global Severity Index, which had 70.0% sensitivity and 77.8% specificity. Because emerging adults in higher education settings are a fascinating population, at risk of elevated distress and psychopathological outcomes, we hope these findings can contribute to better understanding mental health status in emerging adulthood as well as enhancing our psychotherapeutic and counseling services to this population.