Portuguese Society of Intensive Care Score for Predicting SARS-CoV-2 Infection Applied to Inpatients with Pneumonia: A Reliable Tool?

Objectives: Early identification of patients with the novel coronavirus in-duced-disease 2019 (COVID-19) and pneumonia is currently challenging. Few data are available on validated scores predictive of Severe Acute Respiratory Syndrome Coronavirus-2 (SARS-CoV-2) infection. The Portuguese Society of Intensive Care (PSIC) proposed a risk score whose main goals were to predict a higher probability of COVID-19 and optimize hospital resources, adjusting patients’ intervention. This study aimed to validate the PSIC risk score applied to inpatients with pneumonia. Methods: A retrospective analysis of 207 patients with pneumonia admitted to a suspected/confirmed SARS-CoV-2 infection specialized ward (20/03 to 20/05/2020) was performed. Score variables were analyzed to determine the significance of the independent predictive variables on the probability of a positive SARS-CoV-2 rRT-PCR test. The binary logistic regression modeling approach was selected. The best cut-off value was obtained with the Receiver Operating Characteristic (ROC) curve together with the evaluation of the discriminatory power through the Area Under the Curve (AUC). Results: The validation cohort included 145 patients. Typical chest computed-tomography features (OR, 12.16; 95% CI, 3.32 COVID-19 stratification in inpatients with Conclusions: The application of the score to inpatients with may be of value in the of COVID-19. Further this score Respiratory Diabetes, COPD, and/or cardiovascular disease hypertension ischemic antigen


Introduction
Coronavirus induced-disease 2019 (COVID-19) was first reported on the 31st of December of 2019 [1]. Since then, it became a pandemic and it brought a heavy burden for the health system of several countries [2]. Symptoms of COVID-19 are non-specific, and its presentation can range from no symptoms to severe acute respiratory syndrome and death [3]. Widespread testing is essential. However, a single negative test does not exclude COVID-19, especially in highly suspected patients [2] [4]. For a negative test, there are two key factors: pretest probability and test sensitivity. The sensitivity rate of the rRT-PCR is estimated to be 66% -80% [2] [4]. Pretest probability depends on several factors, including local COVID-19 prevalence, exposure history and symptoms [4]. Prediction models that combine several of these features to estimate the risk of people being infected could assist medical staff [2]. Efficient diagnosis tools are necessary to help triage patients when allocating limited healthcare resources [2].
In March 2020, the Portuguese Society of Intensive Care (PSIC) proposed a risk score based on 10 variables to stratify patients with pneumonia regarding Severe Acute Respiratory Syndrome Coronavirus-2 (SARS-CoV-2) infection probability [5]. It confers a relative score to 10 variables, including demographic, clinical, analytical and imaging data ( Table 1). It is considered indicative of a

Methods
This retrospective study was conducted in a COVID-19 dedicated Department,

Statistical Methods
The logistic regression modeling approach was selected considering the binary nature of the dependent variable. The selection of significant independent variables, with predictive power, was done by using the Forward Stepwise (Like- Since the total PSIC score was calculated for each patient and this was also included on our database, in our study two candidate logistic regression models were developed. The probability of a positive SARS-CoV-2 rRT-PCR test was computed. Model I considered the total PSIC score of each patient and Model II considered all the PSIC score variables regardless total score.
Descriptive statistics were performed for the basic analysis of data. Hypothesis tests of significance of the differences between proportions and means of two groups were also performed, assuming, respectively, Bernoulli and Normal populations with unknown equal variances. All the analysis was carried out using the software IBM SPSS Statistics 25.

Results
A total of 207 patients with pneumonia were admitted to the COVID-19 Department. The validation cohort included 145 patients, and 62 (30.0%) were excluded due to lack of information on PSIC score variables such as: chest CT (n = 20), procalcitonin level (n = 40), urinary antigen tests (n = 15), LDH level (n = 3) and information regarding exposure to a positive SARS-CoV-2 patient (n = 1).   Table 2.
PSIC score variables and mean total PSIC scores were compared between the positive and negative COVID-19 groups (Table 3). Typical chest CT features, exposure to a positive COVID-19 patient and PCT < 0.5 ng/mL were significantly more prevalent in the positive SARS-CoV-2 rRT-PCR group (p < 0.0001, <0.0001 and 0.044, respectively). The mean PSIC score was also significantly higher in the positive group (13.17 ± 3.72 vs 8.39 ± 3.67, p < 0.0001).
For Model I, the probability to be positive as a function of the PSIC score is given by the relation and for a cut value between 0.20 and 0.24, sensitivity and specificity were, 87.0% and 69.7%, respectively, with an overall classification performance of 72.4%. For all models, the AUC was greater than 0.8, meaning that their discriminating power can be considered as good.
Further analysis revealed that the best cut-off value for the PSIC score is 10 or 11. The selection of each one depends on what is more relevant-sensitivity (10) or specificity (11) (Figure 1 and Figure 2).

Discussion
Less expensive and complex COVID-19 pneumonia diagnostic methods are ur-   L. Wynants et al. [2] published a metanalysis with critical approach to models published to support the diagnosis of COVID-19 in patients with suspected infection. One group of authors [7] [11]. However, other respiratory pathogens can also present a higher incidence of severe disease in the elderly subpopulations with comorbidities [12] [13]. Indeed, when comparing positive and negative SARS-CoV-2 rRT-PCR subgroups, male gender and comorbidities were equally found in both conditions.
According to recent publications, the most common comorbidities in COVID-19 patients are hypertension, obesity and diabetes [9] [14] [15], which is also in agreement with our findings. Of note, obesity was not analyzed in this study since it was not included in the PSIC score.
When comparing positive and negative SARS-CoV-2 rRT-PCR subgroups, typical chest CT features, exposure to a positive SARS-CoV-2 patient and PCT < 0.5 ng/mL were significantly more prevalent in the positive SARS-CoV-2 rRT-PCR subgroup.
The logistic regression analysis revealed that typical chest CT features and contact with a positive case were the most significant independent variables in predicting a positive rRT-PCR-SARS-CoV-2 test, which partially agrees with the Lymphopenia has been associated with severe coronavirus disease [16] [17].
Nevertheless, its role in predicting infection may be somewhat limited since in this study, the presence of lymphopenia (<1200/uL) didn't determine a significantly increased risk of a positive SARS-CoV-2 test.
The aged study population and the development of lymphopenia in other viral infections and medical conditions may have accounted for these findings since ≥50% of patients in both groups presented this condition. This suggests that the relative scoring system proposed by the PSIC might need adjustment in the weight of this variable for this subpopulation. Only a small percentage of patients tested positive for other respiratory viruses and presented positive urinary antigen tests and the prevalence of negative results was similar between the positive and negative rRT-PCR-SARS-CoV-2 subgroups, which seems to preclude these variables from being good infection discriminators. However, these can be useful tools for diagnosing co-infection, which may imply different treatment strategies. Importantly, in our study, co-infection was found in 39.0% of patients, including other respiratory viruses and bacteria. This finding may have also contributed to the non-significant results in CRP, PCT and LDH levels.
According to our results, the best score cut-off value was between 10 and 11, which is in accordance with the one proposed by the original authors. In this study, most (86.9%) of the positive SARS-CoV-2 patients presented a score ≥ 10 and the mean PSIC score was significantly higher in the confirmed COVID-19 subgroup.
The ROC curve analysis illustrated that the PSIC score can accurately stratify patients with pneumonia in different COVID-19 risk categories.
Although this model has been shown to be a useful risk assessment tool for COVID-19 pneumonia, it may miss-label some patients. Indeed, forty-one (33.6%) patients of the negative COVID-19 subgroup presented a PSIC score ≥ 10, but the meaning of these findings may be somewhat misleading considering the sensitivity of the rRT-PCR test.
Furthermore, the PSIC score was proposed with no description of the methods used for the determination of score variables, grade scoring system and score cut-off, probably due to the need of fast interventions in the setting of a public health emergency.
This study presented several limitations. First, it was a retrospective analysis.
Secondly, it included a limited number of patients reflecting the reality of a single-center department. Not all patients admitted to the ward were included in the validation cohort. Most patients were excluded due to lack of chest CT images since not all were considered to present criteria for the performance of a Open Journal of Respiratory Diseases CT-scan when the clinical picture and the chest radiography were sufficiently informative and the CT radiation risks and costs were considered to outcome its benefits. Other patients were excluded due to a lack of information on other variables such as PCT and urinary antigen tests. However, it should be noted that the study was developed during the initial phase of COVID-19 in Portugal when diagnostic protocols were still being implemented and the exams were requested according to clinical presentation. Results in this study rely on the SARS-CoV-2 rRT-PCR test for the confirmation of positive COVID-19 cases. However, this test presents a sensitivity of approximately 66% -80% [2] [4], which means that a considerable number of positive cases may have not been identified. Moreover, collected samples for the SARS-CoV-2 test were from Naso and oropharyngeal swabs, which present a lower diagnostic yield compared to lower airway samples [10]. The authors hypothesized whether some of these patients could have been good candidates for the serologic monitoring during the acute phase and follow-up, particularly the ones with typical CT features. Regardless of the limitations, this is an attempt to validate a useful tool in the clinical practice with encouraging results, particularly when allocating patients at admission.
In this study, the PSIC score was a reliable and valid tool for assessing the probability of a positive SARS-CoV-2 rt-PCR test in inpatients with pneumonia.
Findings suggest that slight adjustments of the PSIC score might be considered in this subpopulation. When dealing with a highly contagious respiratory virus, the use of diagnostic clinical risk scores coupled with laboratory tests at admission may facilitate hospital patient allocation while under observation promoting intra-hospital infection control. To validate the proposed risk score at a broader level and to improve it, it should be applied at a multicentric scale. So far, proposed models are poorly reported and at high risk of bias. Hence, they cannot be recommended in current practice.

Contributions
VC conceptualized the study. AA collected and organized the data. FD, VD, AA and DN performed the statistical analysis. AA, DN, VD and VC wrote the manuscript. All authors reviewed the manuscript.