Effectiveness of Social Cognition and Interaction Training (SCIT) in Community Settings in Portugal ()
1. Introduction
Social difficulties are a core feature of schizophrenia that are not improved by medication [1] [2] and are only modestly improved by psychological interventions [3]. Social cognition, which refers to the mental operations underlying social interaction [4], is seen as a promising treatment target for improving social functioning. Specifically, social cognitive domains, including Theory of Mind and attributional style, are correlated with social functioning and have been shown to be responsive to treatment intervention in schizophrenia [5] [6] [7].
The most widely studied social cognitive treatment for schizophrenia is Social Cognition and Interaction Training (SCIT) [8], a 24-week group therapy and skills training approach. Although SCIT has been implemented in nine countries with over 1.500 patients, few trials of the intervention have been conducted outside of the United States. Given that social cognitive norms are known to vary across cultures [9], it is important to assess SCIT’s acceptability and effectiveness prior to using it as part of regular treatment programming outside of the United States. Further, despite support in research efficacy trials, few studies have examined whether SCIT leads to measurable improvements in social cognition and functioning when implemented in routine clinical settings by clinicians rather than research staff [10]. The primary goal of the current study was to evaluate SCIT’s effectiveness when implemented in routine outpatient settings in Portugal.
An additional challenge to dissemination of social cognition training programs is the dearth of measures of social cognition that are psychometrically sound and feasible for use in routine clinical settings. Recently, the SCOPE study [11], has been initiated to identify and refine the most promising measures for social cognitive treatment outcome research in psychosis. SCOPE identified the Hinting task [12] as the most promising measure for the assessment of Theory of Mind, but did not find any measures of attributional bias to be psychometrically acceptable. Thus, a secondary goal of this study is to describe the performance of translated versions of the Hinting task and the Ambiguous Intentions Hostility Questionnaire (AIHQ) [13], which is the most widely used measure of hostile attributional bias in social cognitive clinical trials.
The current study used a quasi-experimental trial design to compare treatment-as-usual (TAU) to SCIT plus TAU among individuals receiving treatment for schizophrenia in two community clinics in Portugal. Practicing clinicians implemented SCIT and conducted outcome assessments in the context of routine clinical care. We predicted that SCIT participants would show greater treatment-related effects in social cognition and social functioning than TAU participants.
2. Material and Methods
2.1. Participant Recruitment and Sample Characteristics
Thirty-five adults with medical chart diagnoses of schizophrenia and without current substance use problems were recruited from two community outpatient services in the city of Porto. All participants were receiving regular outpatient psychiatric treatment, including antipsychotic medication, throughout the study. No additional inclusion or exclusion criteria were used. Participants were assigned to the TAU group who either 1) declined to participate in the SCIT group (n = 5) or 2) were unable to attend SCIT due to a scheduling conflict (n = 11), resulting in a TAU group of 16 and a SCIT group of 19. Two SCIT treatment cohorts were conducted, each with 6 to 13 group participants and two co-facilitators.
2.2. Measures
Participants’ diagnoses were obtained from their medical charts. Symptomatology was assessed with the Portuguese version of the Positive and Negative Syndromes Scale (PANSS) [14]. Participants completed assessments at baseline, and then again after six months of treatment as usual or SCIT.
Theory of Mind was measured with the Hinting task [12]. The Hinting task provides participants with ten brief social vignettes in which one character utters an indirect request (or “hint”) to another character. The participant must infer the intended meaning of the hint. Total score on the task ranges from 0 to 20, with higher scores indicating better performance. The SCOPE study found the American language version of the Hinting task to have acceptable test-retest reliability (Pearson r = 0.639), to be strongly correlated with three measures of functional outcome, and to significantly differentiate schizophrenia from healthy control participants. The Hinting task was recommended by SCOPE as the preferred measure of ToM for clinical trials [11].
Hostile attributional bias was measured with the Ambiguous Intentions Hostility Questionnaire-Ambiguous items (AIHQ-A) [13]. The AIHQ-A presents participants with five vignettes describing mildly negative and ambiguous interactions between the participant and other people (e.g., “You walk past a group of teenagers and they start to laugh.”). Participants answer questions about the vignette, yielding five scores—hostility, intentionality, blame, anger, and aggressiveness. Scores are summed to yield a total score ranging from 25 to 125 with higher scores indicating greater hostile attributional bias. In SCOPE, an expert consensus panel selected the AIHQ as the only measure of attributional bias for consideration. However, the study found that the AIHQ’s test-retest reliability was unacceptable, the scale was not related to functional outcome, and its ability to differentiate patient and control groups was inconsistent.
Social Functioning was measured with the Social Skill Performance Assessment (SSPA) [15] and the Social Functioning Scale (SFS) [16]. The SSPA requires the participant to complete two role-plays with a confederate, one in which they negotiate with a landlord to get a leak fixed and the other in which they get to know a new neighbour. Performance is rated across multiple domains, including social appropriateness, speech clarity, affect, and effectiveness. SSPA total scores range from 16 to 80, with higher scores indicating better interpersonal effectiveness. The SFS is an interview-based survey of the participant’s functioning in a range of social and vocational spheres, including social engagement, communication, performance of independent living skills, and educational/vocational achievement. Total Score ranges from 0 to 208, with higher score indicating greater social functioning.
The translation of Hinting Task, AIHQ, SSPA and SFS followed a common procedure: 1) Two bilingual translators independently translated the items from the original English version to Portuguese; 2) The translators discussed any inconsistencies, and a composite translation was produced; 3) This composite translation was back-translated to English by an independent translator whose mother tongue is English and who had no knowledge of the instruments; 4) The original and the back-translation were compared, and translators agreed on the final version. All symptom, social cognition and social functioning measures were administered to participants by treating clinicians.
2.3. Treatment Conditions
Treatment as usual (TAU) consisted of the suite of services available in the two participating community clinics, including medication management, individual psychotherapy and occupational activities/therapy. The exact combination of TAU services differed across participants, however all participants received medication management and a combination of occupational therapy and/or individual psychotherapy.
SCIT was delivered following the treatment manual [8] by clinicians who participated in a day-long training with one of the SCIT treatment developers. SCIT combines psychoeducation, group discussion, and skill practice in the areas of Theory of Mind, attributional bias, jumping to conclusions, and emotion perception. Minor adaptations were made to address language/cultural differences and constraints of the treatment environment. Specifically, videos and verbal content were translated into Portuguese. The number of sessions provided across the two cohorts was 22.
3. Results
All 35 study participants completed baseline assessments, but five dropped out of SCIT treatment during Phase 1 (one could not be in the group due to psychotic symptoms; one believed he didn’t need; one started another activity; one due to transportation issues; and one just never showed up again), leaving a total of 30 study completers. Fourteen of 19 patients completed SCIT therapy (73.68), defined as attending more than half of the sessions, including at least two sessions at each phase.
3.1. Demographic and Baseline Clinical Analyses
Table 1 summarizes the baseline clinical and demographic characteristics for both groups. T-tests revealed that the SCIT and TAU groups did not differ significantly on any demographic variables or any baseline symptomatology, social cognition or social functioning variables.
![]()
Table 1. Demographic and clinical information.
SCIT = Social Cognition and Interaction Training; TAU = Treatment as Usual; PANSS = Positive and Negative Syndromes Scale.
3.2. Primary Analyses
Effects of treatment on social cognition and social functioning were examined using mixed ANOVAs with Time (pre- vs. post-treatment) as a within-subjects factor and Group (SCIT vs. TAU) as a between-subjects factor. Significant interactions were probed with paired-samples t-tests. Effect sizes for within-group changes were computed using Dunlap and colleague’s [17] method.
Treatment effects on are summarized in Table 2. On the Hinting task, the main effect for Time was statistically significant (F = 11.86; p = 0.002), and the Time X Group interaction was significant at a trend level (F = 3.57; p = 0.069). Probing of the interaction revealed a statistically significant improvement in both the SCIT (t = 2.65; p = 0.020) and TAU groups (t = 2.42; p = 0.029), with a larger effect size in the SCIT group (d = 0.42 vs. 0.16). Because previous studies (e.g., Roberts & Penn, 2009) have observed ceiling effects on the Hinting task, we examined this possibility. Seventeen percent (5/30) of our participants received perfect scores on the Hinting task at baseline, while 50% (15/30) performed at or above the SCOPE study normative mean of 17 at baseline.
On the AIHQ-A, the main effect for Time was statistically significant (F = 4.76; p = 0.038) and the Time X Group interaction was significant at a trend level in the expected direction (F = 3.97; p = 0.056). Probing revealed a trend level improvement in the SCIT group (t = 1.97; p = 0.070), at a small-to-medium effect size (d = 0.36) and no change in the TAU group. Because some evidence suggests that hostile attributional bias is only present in a subgroup of people who have schizophrenia [18] [19] we conducted a post-hoc analysis in which we
![]()
Table 2. SCIT vs. TAU treatment effects.
SCIT = Social Cognition and Interaction Training; TAU = Treatment as Usual. f Time × Group interaction: p < 0.07; ** Time x Group interaction: p < 0.001.
re-ran the mixed ANOVA only within the subset of patients whose baseline AIHQ-A score was at or above the overall baseline mean of 65. This analysis yielded a statistically significant Time X Group interaction (F = 5.39; p = 0.037), with probing showing that SCIT participants’ (n = 9) scores decreased significantly from 80.11 to 70.56 (t = 2.96; p = 0.018) with a large effect size (d = 0.96), whereas the TAU group’s mean did not change from baseline (78.33) to post-treatment 78.17 (NS).
On the Social Skill Performance Assessment, the main effect for Time was statistically significant (F = 30.615; p < 0.001), and the Time X Group interaction was significant in the expected direction (F = 16.451; p < 0.001), driven by highly significant improvement within the SCIT group (t = 4.55; p < 0.001) that reached a moderate to large effect size (d = 0.59). Finally, on the Social Functioning Scale, neither the main effect for Time nor the Time X Group interaction was statistically significant, indicating that neither group showed changes in this domain during the treatment period.
4. Discussion
This is the second study that has evaluated SCIT’s effectiveness when implemented by practicing clinicians, in routine clinical settings, and with clinicians conducting outcome assessment [10]. This study also examined the use of translated treatment materials and social cognitive outcome measures. We hypothesized that community-dwelling people with schizophrenia who received a Portuguese version of SCIT administered by community clinicians would experience improved social cognition and social functioning relative to a TAU group. Hypotheses were largely supported, however conclusions must be tempered due to methodological and measurement limitations.
Regarding Hinting task (ToM) findings, both the SCIT and TAU groups showed significant improvement, with the SCIT group showing a notably larger effect size (0.42 vs. 0.16). Thus, it is likely that SCIT improved ToM, however it is possible that results were affected by rater bias among the assessing clinicians. It is also possible that both SCIT and TAU group improvements are due largely to practice effects on the instrument. The SCOPE trial did also observe small practice effects on the American Hinting task [11], thus, apparent practice effects in the present study support the convergence of the Portuguese language task with the American. Our Portuguese Hinting task data also mirrored data from American studies [20] in the presence of ceiling effects. We found that fully half of our participants scored at the normative mean (17 out of 20) at baseline, while 17% received perfect scores at baseline. These findings substantially limit our ability to accurately assess the extent to which SCIT improved ToM in this sample. In the SCOPE study, only 7% of schizophrenia participants were found to score at ceiling on the Hinting task [11], however ceiling was operationalized conservatively as scoring at 100%. We consider normative performance to be a more ecologically valid measure of ceiling effect as it captures patients whose performance cannot be distinguished from normative performance. Our findings reinforce the conclusion that the sensitivity of the Hinting task may be low among higher functioning patients. This has important implications for social cognition trials among individuals with first episode or attenuated psychosis, who often are higher functioning. Unfortunately, sample size in the current study was too small to examine Hinting task effects within the subgroup of participants who scored below the normative mean at baseline. We suggest that future social cognitive trials using the Hinting task consider defining ceiling effects normatively rather than based on perfect performance, consider screening for ceiling effects, routinely examine for ceiling effects during data analysis, and use sample sizes large enough to conduct subgroup analyses when ceiling effects are detected.
AIHQ-A findings were promising in that the SCIT group showed trend-level decreases in bias while the TAU group showed no change. Among participants with elevated baseline AIHQ-A, this difference became statistically significant and reached a large within-group effect size among SCIT participants. Although conclusions must be tempered due to the AIHQ-A being administered by treating clinicians, these findings provide tentative support for the effectiveness of SCIT at decreasing hostile attributional bias, particularly within the subset of patients who have elevated bias relative to other patients. The notion that dysfunctional attributional style may be present only within a subset of patients with schizophrenia has been supported in previous research showing that biases may be influenced by affective states and thinking styles, as seen in depressive attributional style [18]. Our current subgroup findings may help to account for the particularly inconsistent evidence of treatment effects on attributional bias in previous studies of SCIT and related interventions [5]. As with the Hinting task, the current AIHQ-A finding suggests that the benefits of interventions such as SCIT may be most detectable among people with demonstrable types of social cognitive dysfunction, rather than among people with schizophrenia as a whole. Future trials using attributional bias as an outcome may do well to follow the lead of Fiszdon and colleagues [21] who specifically recruited participants with baseline elevations in attributional bias.
Social skill performance (SSPA) results strongly supported SCIT’s effectiveness in improving an ecologically valid social functioning domain. This is quite promising as the ultimate aim of SCIT is to improve social functioning. However, the fact that the SSPA in this study was both administered and scored by participants’ treating clinicians creates a substantial risk of rater bias, and thus the validity of this result must remain uncertain. Finally, the lack of support for SCIT on the SFS is disappointing. One explanation for the contrast between the SSPA and SFS findings is that the latter measures treatment-distal social outcomes (e.g., vocational attainment, number of social encounters per week) which would be expected to change more slowly (over the course of months) relative to social role-play performance which gives participants a here-and-now opportunity to demonstrate newly acquired skills, and thus may yield more immediately detectable effects as a result of social cognitive treatment.
This study has several notable limitations, including non-blinded assessment and small sample size. Non-blinded assessment is a typical feature of outcome assessment in routine clinical settings, but nonetheless it leaves open substantial risk for bias, especially on the Hinting Task, AIHQ, and SSPA, which require subjective judgment in coding of participant responses. In the case of the Hinting task, this risk arguably is overshadowed by widespread baseline ceiling effects which constrained our ability to detect improved scores. Small sample size is problematic in this study not only because it limits reliability of data and potential to detect effects, but also because it limited our ability to conduct subgroup analyses. In particular, it was not possible to conduct statistical tests within the subgroup of participants who did not exhibit Hinting task baseline ceiling effects.
In summary, the current study supports the feasibility and potential effectiveness of SCIT in Portuguese community settings. The Hinting task and AIHQ performed similar to the English language versions of these tasks, which enables cross-country comparison of results, but also unfortunately recapitulates known psychometric challenges with two of the most commonly used outcome measures in social cognitive treatment trials. While work continues to improve social cognition measurement, we suggest that future trials using these tasks consider either screening for participants who score in the dysfunctional range at baseline or recruiting large enough samples to allow for subgroup analyses.
Acknowledgements
The project of adapting SCIT to the Portuguese population represents an effort undertaken by ENCONTRAR+SE—Association for the promotion of mental health (NGO) to bring to Portugal the best/most promising practices to empower people with mental health problems.
The authors are grateful to the Fundação para a Ciência e Tecnologia (FCT) for the support to this publication (Ref. **UID/CED/4872/2019).
The authors wish to thank the support of David Roberts, one of the authors of SCIT throughout the project. We would also like to thank the institutions where the study took place, the clinical psychologists Carmina Rei, Natália Costa, Ana Morais, Mariana Moreira e Patrícia Gomes who carried out the fieldwork; the clinical psychologist Marta Magalhães Basto who collaborated in the evaluation process; and the mental health professionals from Hospital Professor Doutor Fernando da Fonseca, coordinated by Dr. Teresa Maia, who helped in the adaptation and translation of SCIT to Portuguese language. Last but not least, the authors are immensely thankful to the service-users who collaborated in the study.