ABSTRACT

Objective: To compare the performance of Charlson Comorbidity Index (CCI) with those of the mental Confusion, Urea, Respiratory rate, Blood pressure, and age = 65 years (CURB-65) score and the Pneumonia Severity Index (PSI) as predictors of all-cause in-hospital mortality in patients with community-acquired pneumonia (CAP). Methods: This was a cohort study involving hospitalized patients with CAP between April of 2014 and March of 2015. Clinical, laboratory, and radiological data were obtained in the ER, and the scores of CCI, CURB-65, and PSI were calculated. The performance of the models was compared using ROC curves and AUCs (95% CI). Results: Of the 459 patients evaluated, 304 met the eligibility criteria. The all-cause in-hospital mortality rate was 15.5%, and 89 (29.3%) of the patients were admitted to the ICU. The AUC for the CCI was significantly greater than those for CURB-65 and PSI (0.83 vs. 0.73 and 0.75, respectively). Conclusions: In this sample of hospitalized patients with CAP, CCI was a better predictor of all-cause in-hospital mortality than were the PSI and CURB-65.

Keywords: Pneumonia, ROC curve; Predictive value of tests; Severity of illness index.

RESUMO

Objetivo: Comparar o desempenho do Índice de Comorbidade de Charlson (ICC) com o do mental Confusion, Urea, Respiratory rate, Blood pressure, and age = 65 years (CURB-65, Confusão mental, Ureia, frequência Respiratória, Pressão arterial e idade = 65 anos) e do Pneumonia Severity Index (PSI, Índice de Gravidade da Pneumonia) como preditores de mortalidade hospitalar por qualquer causa em pacientes com pneumonia adquirida na comunidade (PAC). Métodos: Estudo de coorte com pacientes hospitalizados com PAC entre abril de 2014 e março de 2015. Dados clínicos, laboratoriais e radiológicos foram obtidos no PS, e o ICC, CURB-65 e PSI foram calculados. O desempenho dos modelos foi comparado por meio de curvas ROC e ASC (IC95%). Resultados: Dos 459 pacientes avaliados, 304 preencheram os critérios de elegibilidade. A taxa de mortalidade hospitalar por qualquer causa foi de 15,5%, e 89 (29,3%) dos pacientes foram admitidos na UTI. A ASC do ICC foi significativamente maior do que a do CURB-65 e do PSI (0,83 vs. 0,73 e 0,75, respectivamente). Conclusões: Nesta amostra de pacientes hospitalizados com PAC, o ICC foi um preditor melhor de mortalidade hospitalar por qualquer causa do que o PSI e o CURB-65.

Palavras-chave: Pneumonia; Curva ROC; Valor preditivo dos testes; Índice de gravidade de doença.

INTRODUCTION
 
Community-acquired pneumonia (CAP) remains to be the leading cause of death from infectious diseases in the world,(1) with an annual incidence of 5-11 cases per 1,000 population, causing major impacts on health care systems.(2) In the USA, more than 60,000 CAP-related deaths were reported among individuals aged ≥ 15 years in 2005, and the annual economic burden was still high in 2010 (17 billion dollars).(3)
 
Early identification of patients at risk of death is a tenet of CAP management, the definition of CAP severity being the most important aspect guiding the decision to hospital admission.(4,5) However, clinical assessment might not accurately capture the severity of the disease and the potential for complications or death.(5,6) As a result, the use of severity scores(7-9) has been recommended to evaluate patients with CAP and to establish the need for intensive care.
 
Among the best known CAP risk prediction models, the mental Confusion, Urea, Respiratory rate, Blood pressure, and age = 65 years (CURB-65) score(10) and the Pneumonia Severity Index (PSI),(11) whose predictive capacity for mortality is 0.79 and 0.82, respectively, have been validated for use in a variety of clinical scenarios.(12) However, both of these models rely on pneumonia-specific criteria and, therefore, do not account for risks associated with comorbidities. Nevertheless, previous studies(13,14) have shown that information regarding the number of comorbidities and degree of health status involvement is helpful to establish prognosis. In that situation, a general score such as the Charlson Comorbidity Index (CCI)(15) can be useful. The CCI, which was developed to standardize the assessment of comorbid conditions(13) and 1- and 10-year all-cause mortality,(15) is a well-established predictor of in-hospital mortality in nonsurgical patients(16) and in those with specific diseases.(17) However, the use of the CCI to predict in-hospital mortality in CAP patients, especially as an alternative to pneumonia-specific severity scores, has yet to be investigated. Thus, the objective of the present study was to compare the performance of CCI with those of CURB-65 and PSI as predictors of all cause in-hospital mortality in patients with CAP.
 
METHODS
 
Study population
 
This study was carried out in a 130-bed general community hospital located in the city of Montenegro, state of Rio Grande do Sul, Brazil. The hospital provides public health care through the Brazilian Unified Health Care System to a population of about 160,000 from 19 cities. CAP was the main reason for admission to the hospital, with a mortality rate of approximately 15.5%.(18) At the time the present study was conducted, the hospital was beginning to implement the use of severity indices to assess the need for admission in patients seeking the ER. The health care professionals in charge of collecting standardized data to calculate the indices were trained by the research team. During the training stage, 100% of the assessments were performed in duplicate, which produced an overall inter-rater agreement of 96.3%.
 
Study design
 
In the present cohort study, we evaluated patients ≥ 14 years of age presenting to our ER with respiratory complaints between April of 2014 and March of 2015. Patients with a clinical and radiographic diagnosis of CAP requiring hospitalization were included in the study. We excluded patients with hospital-acquired pneumonia (characterized by admission to urgent-care facilities for at least 2 days); patients originating from retirement homes, shelters, or other health care institutions; patients on intravenous antibiotic treatment or chemotherapy; patients treated for pressure ulcers in the previous 30 days; and patients undergoing renal replacement therapy.
 
The results of severity assessment using the risk prediction models were recorded in the medical charts of the patients and taken as baseline data for the cohort. The clinical progress of patients was assessed during hospitalization. Hospital discharge was defined as the clinical outcome measure.
 
CAP was diagnosed on the basis of at least one of the following chest X-ray findings: new or progressive infiltrate, consolidation, or cavitation; and at least one of the following signs or symptoms: fever > 38°C with no other known cause, leukopenia (< 4,000 leukocytes/mm3), or leukocytosis (≥ 12,000 leukocytes/mm3). In addition, in patients aged ≥ 70 years, changes in mental state with no other evident cause and at least two of the following were considered for the diagnosis of CAP: recent cough with purulent sputum, changes in expectoration, increase in respiratory secretions, increase in the frequency of aspiration, onset or worsening of cough, dyspnea or tachypnea, wheezing, or worsening of gas exchange (for example, oxygen desaturation [PaO2/FiO2 ≤ 240], increased need for oxygen, or need for mechanical ventilation).
 
The study was approved by the institutional review board (Protocol no. 150168).
 
Study variables
 
Clinical, laboratory, and radiological data recorded in the medical chart were obtained in the first 24 h after the ER consultation, including age, sex, origin, RR, blood pressure, temperature, HR, presence of mental confusion, SpO2, comorbidities (added to the medical record by an attending physician), history of hospital admissions, chest X-ray findings (reported by a radiologist), and results of laboratory tests requested during the ER visit. Laboratory tests included arterial blood gas analysis, urea, serum creatinine, glucose, sodium, and blood workup. Information regarding antibiotic treatment duration, length of hospital stay, length of ICU stay, and need for mechanical ventilation was also collected. The major outcome measure was all-cause in-hospital mortality recorded in the medical chart and confirmed through review of the discharge summary or of the death certificate accordingly. This information was available for all CAP patients treated at the hospital during the study period. The discharge summary was prepared by an attending physician after discharge in all cases.
 
For the analysis, patients were grouped into two categories—low risk or intermediate/high risk—according to the cutoff point of each model: the CCI(15) covers 19 variables related to comorbidities, with scores ranging from 1 to 6, patients with a CCI of 0-2 being classified as at a low risk of death/admission; CURB-65(10) is based on the assessment of five clinical characteristics, with scores ranging from 0 to 5, patients with a CURB-65 of 0 or 1 being classified as at a low risk of death/admission; the PSI(11) relies on 20 clinical variables to generate a score with five classes representing progressive increase in the risk of death, patients with a PSI score of 1 or 2 being classified as at a low risk of death.
 
Data analysis and sample size calculation
 
In order to determine the capacity of CCI, CURB-65, and PSI to predict the risk of death, ROC curves and the C statistic (corresponding to the AUC) were used. The measure of calibration used was the Hosmer-Lemeshow test. An AUC of 0.5 indicates no discriminating power, an AUC of 0.7-0.8 indicates clinical usefulness, and values above 0.8 indicate excellent predictive capacity.(19) AUCs were compared using DeLong’s method for CCI vs. CURB-65 and PSI. The comparison among the proportions of patients classified as being at a low risk by the three indices was performed using McNemar’s test. A two-tailed p < 0.05 was considered statistically significant. A bivariate analysis of clinical characteristics vs. mortality was performed using the Student’s t test for means and standard deviations or Pearson’s chi-square test for proportions.
 
All analyses were performed with the SPSS Statistics software package, version 17 (SPSS Inc., Chicago, IL, USA) and Epidat, version 3.0 (Dirección Xeral de Saúde Pública de la Consellería de Sanidade, Xunta de Galicia, Santiago de Compostela, Spain).
 
The sample size was calculated using a simulation approach, considering differences between the scores in terms of sensitivity (ranging from 75% to 95%), specificity (from 50% to 70%), a survival:death ratio of 6:1, a statistical power of at least 80%, and a 95% CI. The resulting sample size was 304 patients.
 
RESULTS
 
Between April of 2014 and March of 2015, 459 patients with respiratory infections were evaluated. Of those, 155 did not meet the diagnostic criteria for CAP, and 304 were enrolled in the study (Figure 1). The mean age of the participants was 67.1 ± 17.3 years, 210 (69.0%) lived in urban areas, 171 (56.3%) were male, and 149 (49.0%) had asthma or COPD as a pre-existing lung disease. During the follow-up period, 47 patients (15.5%) died, 89 (29.3%) were admitted to the ICU, and 98 (32.2%) required mechanical ventilation (Table 1).


 
Clinical examination revealed that approximately one-third of the participants had airway secretions, and sputum was collected. Specimens for culture (sputum or blood) were collected from 203 patients (66.8%), and infectious agents were isolated in 52. The most common infectious agent was Streptococcus pneumoniae, in 19 patients (36.5%). Treatment was based on amoxicillin-clavulanate (72.2%) and/or azithromycin (65.6%). Mean duration of hospital stay was 7.2 ± 7.4 days (median, 5.0 days).
 
Table 2 shows that the scores of the three risk prediction models increased linearly with the increase in the mortality rate. The number of patients considered to be at a low risk according to the CCI, CURB-65, and PSI were 74 (24.3%), 89 (29.3%), and 80 (26.3%), respectively. The death rate of patients classified as being at a low risk by the CCI, CURB-65, and PSI was low (1.4%, 4.5%, and 3.7%, respectively).
 





 
Figure 2 shows that CCI was an excellent predictor of all-cause in-hospital mortality, with a greater AUC (0.83) than those for curb-65 (0.73; p = 0.02) and PSI (0.75; p = 0.04). There was no statistical difference between the AUCs of CURB-65 and PSI (p = 0.7). After Hosmer-Lemeshow calibration, p values for CCI, PSI, and CURB-65 were 0.9988, 0.9769, and 0.9906, respectively.

 
An analysis of sensitivity comparing patients with and without previous lung disease did not reveal differences among the models to predict in-hospital mortality. The CCI for patients without previous lung disease (0.86; 95% CI: 0.78-0.93) was similar to that for those with previous lung disease (0.82; 95% CI: 0.73-0.91).
 
In our study, we decided not to exclude patients with a do-not-resuscitate order (n = 29), and 24 of those patients died. When we excluded those patients, there were no important changes in the AUCs (CCI = 0.83; CURB-65 = 0.75; and PSI = 0.74).
 
DISCUSSION
 
The present study using the C statistic showed that the CCI performed better than did CURB-65 and PSI to predict all-cause in-hospital mortality in patients admitted for CAP. To the best of our knowledge, this was the first study assessing the CCI as a predictor of all-cause in-hospital mortality in patients with CAP spontaneously seeking emergency care at a community hospital over a period of 1 year.
 
A previous study comparing the CCI with CURB-65 and PSI enrolled only elderly hospitalized individuals with pneumonia. The study did not detect a statistical difference between mortality prediction scores over 1 year.(20) The AUCs observed in the present study are similar to those previously described for CURB-65 (0.73 to 0.76) and PSI (0.70 to 0.80).(11,21-24) It is important to note that the scores do not measure the same construct. The CCI is a comorbidity score, with several variables. Unlike the CCI, CURB-65 and CRB-65 (no measurement of urea) scores are viewed as markers of disease severity at admission that are similar to PSI. Our findings support the notion that, despite being a general score, the CCI has an excellent predictive performance in patients with CAP.
 
The number of variables covered by a score can be associated with its overall performance; nevertheless, despite including a similar number of variables, the CCI and PSI differ regarding comorbidities, which are covered by the CCI, whereas PSI only accounts for pneumonia-specific characteristics. We confirmed the high sensitivity of CCI and found a low proportion of CAP patients who received a low-risk CCI and died (1.4%). These findings suggest that the CCI has more potential for clinical use than does the PSI or CURB-65.
 
The use of risk prediction models is warranted by guidelines for CAP management.(7-9) However, the detection of CAP severity is usually determined by clinical assessment,(21) which is frequently performed without the support from an objective, structured tool.(22) In this sense, the CCI has the advantage of being part of the usual assessment of severity in emergency services and, consequently, does not need to be introduced in the routine of patient care for the assessment of individuals with pneumonia. In addition, since the CCI does not require laboratory tests, it is appropriate for use in emergency settings. Finally, the CCI has been validated in a variety of clinical scenarios, and the results obtained so far consistently show that the CCI is a good predictor of mortality.(16,23,24) In the present study, the sensitivity analysis showed that the CCI had a prognostic performance that was similar in patients with and without previous lung disease.
 
The results of the present study must be interpreted in light of some potential limitations. All study participants were enrolled in one single center in a mid-sized city, which could limit the generalizability of the findings to a certain extent. Conversely, it is likely that all eligible patients were included, since the community hospital is the only institution where patients with CAP can be hospitalized in that geographic area. Another positive aspect is that, during the study, the CCI was being assessed as an institutional strategy for decision-making regarding hospital admission. This translated into institutional engagement, standardization of clinical assessment, and design of clinical forms for data collection to be adopted by the ER. As a result, there were no losses to follow-up and the information collected had high quality, both of which are strengths of this study. Finally, similarly to all studies with a retrospective design, there are possible limitations, such as confounding and information biases. However, we do not believe that this affected the validity of our findings. The data in use were mainly assessed and documented during the hospital stay of the patients.
 
Another point that should be emphasized is that our results could not be generalized to the outpatient population. Patients admitted with CAP have their own characteristics, older age being one of the most important ones. The mean age of the patients in our study was 67 years, and only 24 patients were younger than 40 years of age. Data in the literature suggest that PSI has poor performance in younger patients,(25,26) and it is possible that the same occurs with the CCI. Due to the small number of deaths in younger patients (only 1), it was not possible to make this kind of assessment in the present study.
 
In conclusion, the present study showed that the CCI, when compared with PSI and CURB-65, is a better predictor of all-cause in-hospital mortality in patients with CAP. Using the CCI in ERs might contribute to reducing the mortality of patients with CAP.
 
AUTHOR CONTRIBUTIONS
 
LFB: study conception and design, interpretation of data, drafting and revision of the manuscript, and approval of the final version; LPD and SCF: study conception and design, drafting and revision of the manuscript, and approval of the final version.
 
REFERENCES
 

1. World Health Organization [homepage on the Internet]. Geneva: World Health Organization; [cited 2020 Sep 1]. Causes of death: Ten leading causes of death, 2012. Available from: http://apps.who.int/gho/data/view.wrapper.MGHEMORTCAUSE10?lang=en.


2. Marrie TJ, Huang JQ. Epidemiology of community-acquired pneumonia in Edmonton, Alberta: an emergency department-based study. Can Respir J. 2005;12(3):139-142. https://doi.org/10.1155/2005/672501


3. File TM Jr, Marrie TJ. Burden of community-acquired pneumonia in North American adults. Postgrad Med. 2010;122(2):130-141. https://doi.org/10.3810/pgm.2010.03.2130


4. Myint PK, Sankaran P, Musonda P, Subramanian DN, Ruffell H, Smith AC, et al. Performance of CURB-65 and CURB-age in community-acquired pneumonia. Int J Clin Pract. 2009;63(9):1345-1350. https://doi.org/10.1111/j.1742-1241.2009.02147.x


5. Neill AM, Martin IR, Weir R, Anderson R, Chereshsky A, Epton MJ, et al. Community acquired pneumonia: aetiology and usefulness of severity criteria on admission. Thorax. 1996;51(10):1010-1016. https://doi.org/10.1136/thx.51.10.1010


6. Woodhead MA, Macfarlane JT, McCracken JS, Rose DH, Finch RG. Prospective study of the aetiology and outcome of pneumonia in the community. Lancet. 1987;1(8534):671-674. https://doi.org/10.1016/S0140-6736(87)90430-2


7. Mandell LA, Wunderink RG, Anzueto A, Bartlett JG, Campbell GD, Dean NC, et al. Infectious Diseases Society of America/American Thoracic Society consensus guidelines on the management of community-acquired pneumonia in adults. Clin Infect Dis. 2007;44 Suppl 2(Suppl 2):S27-S72. https://doi.org/10.1086/511159


8. British Thoracic Society Standards of Care Committee. BTS Guidelines for the Management of Community Acquired Pneumonia in Adults. Thorax. 2001;56 Suppl 4(Suppl 4):IV1-IV64. https://doi.org/10.1136/thx.56.suppl_4.iv1


9. Corrêa Rde A, Lundgren FL, Pereira-Silva JL, Frare e Silva RL, Cardoso AP, Lemos AC, et al. Brazilian guidelines for community-acquired pneumonia in immunocompetent adults - 2009. J Bras Pneumol. 2009;35(6):574-601. https://doi.org/10.1590/S1806-37132009000600011


10. Lim WS, van der Eerden MM, Laing R, Boersma WG, Karalus N, Town GI, et al. Defining community acquired pneumonia severity on presentation to hospital: an international derivation and validation study. Thorax. 2003;58(5):377-382. https://doi.org/10.1136/thorax.58.5.377


11. Fine MJ, Auble TE, Yealy DM, Hanusa BH, Weissfeld LA, Singer DE, et al. A prediction rule to identify low-risk patients with community-acquired pneumonia. N Engl J Med. 1997;336(4):243-250. https://doi.org/10.1056/NEJM199701233360402


12. Kwok CS, Loke YK, Woo K, Myint PK. Risk prediction models for mortality in community-acquired pneumonia: a systematic review. Biomed Res Int. 2013;2013:504136. https://doi.org/10.1155/2013/504136


13. Lu KJ, Kearney LG, Ord M, Jones E, Burrell LM, Srivastava PM. Age adjusted Charlson Co-morbidity Index is an independent predictor of mortality over long-term follow-up in infective endocarditis. Int J Cardiol. 2013;168(6):5243-5248. https://doi.org/10.1016/j.ijcard.2013.08.023


14. Martins M, Blais R. Evaluation of comorbidity indices for inpatient mortality prediction models. J Clin Epidemiol. 2006;59(7):665-669. https://doi.org/10.1016/j.jclinepi.2005.11.017


15. Charlson ME, Pompei P, Ales KL, MacKenzie CR. A new method of classifying prognostic comorbidity in longitudinal studies: development and validation. J Chronic Dis. 1987;40(5):373-383. https://doi.org/10.1016/0021-9681(87)90171-8


16. Sundararajan V, Henderson T, Perry C, Muggivan A, Quan H, Ghali WA. New ICD-10 version of the Charlson comorbidity index predicted in-hospital mortality. J Clin Epidemiol. 2004;57(12):1288-1294. https://doi.org/10.1016/j.jclinepi.2004.03.012


17. Zavascki AP, Fuchs SC. The need for reappraisal of AIDS score weight of Charlson comorbidity index. J Clin Epidemiol. 2007;60(9):867-868. https://doi.org/10.1016/j.jclinepi.2006.11.004


18. Bahlis LF, Diogo LP, Kuchenbecker RS, Fuchs SC. Clinical, epidemiological, and etiological profile of inpatients with community-acquired pneumonia in a public hospital in the interior of Brazil. J Bras Pneumol. 2018;44(4):261-266. https://doi.org/10.1590/s1806-37562017000000434


19. Hanley JA, McNeil BJ. The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology. 1982;143(1):29-36. https://doi.org/10.1148/radiology.143.1.7063747


20. Wesemann T, Nüllmann H, Pflug MA, Heppner HJ, Pientka L, Thiem U. Pneumonia severity, comorbidity and 1-year mortality in predominantly older adults with community-acquired pneumonia: a cohort study. BMC Infect Dis. 2015;15:2. https://doi.org/10.1186/s12879-014-0730-x


21. Salluh JI, Lisboa T, Bozza FA, Soares M, Póvoa P. Management of severe community-acquired pneumonia: a survey on the attitudes of 468 physicians in Iberia and South America. J Crit Care. 2014;29(5):743-747. https://doi.org/10.1016/j.jcrc.2014.05.019


22. Rabello L, Conceição C, Ebecken K, Lisboa T, Bozza FA, Soares M, et al. Management of severe community-acquired pneumonia in Brazil: a secondary analysis of an international survey. Rev Bras Ter Intensiva. 2015;27(1):57-63. https://doi.org/10.5935/0103-507X.20150010


23. Grendar J, Shaheen AA, Myers RP, Parker R, Vollmer CM Jr, Ball CG, et al. Predicting in-hospital mortality in patients undergoing complex gastrointestinal surgery: determining the optimal risk adjustment method. Arch Surg. 2012;147(2):126-135. https://doi.org/10.1001/archsurg.2011.296


24. Budweiser S, Harlacher M, Pfeifer M, Jörres RA. Co-morbidities and hyperinflation are independent risk factors of all-cause mortality in very severe COPD. COPD. 2014;11(4):388-400. https://doi.org/10.3109/15412555.2013.836174


25. Prina E, Ranzani OT, Torres A. Community-acquired pneumonia. Lancet. 2015;386(9998):1097-1108. https://doi.org/10.1016/S0140-6736(15)60733-4


26. Marti C, Garin N, Grosgurin O, Poncet A, Combescure C, Carballo S, et al. Prediction of severe community-acquired pneumonia: a systematic review and meta-analysis. Crit Care. 2012;16(4):R141. https://doi.org/10.1186/cc11447