ABSTRACT

Objective: To evaluate the competence of senior medical students in diagnosing tuberculosis (TB) based on their reading of chest X-rays, as well as to identify the factors associated with high scores for the overall interpretation of chest X-rays. Methods: In October 2008, a convenience sample of senior medical students who had undergone formal training in radiology at the Federal University of Rio de Janeiro School of Medicine, in the city of Rio de Janeiro, Brazil, were invited to participate in the study. Six chest X-rays (three of TB patients and three of patients without TB) were selected. Participants were asked to choose one of the three probable radiological interpretations, and one of the four subsequent suitable clinical approaches. They also completed a questionnaire designed to collect data related to demographics, career of interest, time spent in emergency rooms and year of study. The sensitivity and specificity related to competence in the radiological diagnosis of TB, as well as a score for the overall interpretation of chest X-rays, were calculated. Results: The sensitivity of the probable radiological diagnosis of pulmonary TB, based on the three chest X-rays of patients with TB (minimal, moderate and extensive) was 86.5%, 90.4% and 94.2%, respectively, and the specificity was 90%, 82% and 42%. The only factor associated with a high score for the overall radiological interpretation was the year of study. Conclusions: In this sample of medical students, who had received formal training in radiology early in their medical school course, the competence in interpreting the chest X-rays of TB patients was good. The year of study seems to influence overall chest X-ray reading skill.

Keywords: Tuberculosis, pulmonary; Radiology; Education, medical.

RESUMO

Objetivo: Avaliar a competência de estudantes de medicina seniores na interpretação de radiografias de tórax para o diagnóstico de tuberculose (TB) e determinar fatores associados com altos escores na interpretação de radiografias de tórax em geral. Métodos: Em outubro de 2008, uma amostra de conveniência de estudantes de medicina seniores da Faculdade de Medicina da Universidade Federal do Rio de Janeiro (RJ), que receberam educação formal em radiologia, foi convidada a participar do estudo. Seis radiografias de tórax foram selecionadas, das quais três eram de pacientes com TB. Os participantes escolheram uma entre três possíveis interpretações radiológicas e uma entre quatro condutas clínicas a serem seguidas. Eles também responderam um questionário relativo a dados demográficos, carreira de interesse, tempo de treinamento na emergência e ano de estudo em medicina. A sensibilidade e especificidade para a competência no diagnóstico radiológico da TB, assim como um escore de acertos em radiografia do tórax em geral, foram calculados. Resultados: A sensibilidade para o diagnóstico radiológico provável de TB pulmonar, baseado nas três radiografias de tórax de pacientes com TB (lesões menos extensas, moderadas e mais extensas) foi de 86,5%, 90,4% e 94,2%, respectivamente, e a especificidade foi de 90%, 82% e 42%. O único fator associado a um alto escore no diagnóstico radiológico geral foi o ano de estudo em medicina. Conclusões: A competência na interpretação de radiografias de tórax de pacientes com TB entre esta amostra de estudantes de medicina, que tiveram treinamento formal em radiologia no início do curso médico, foi boa. O ano de estudo médico parece contribuir com a habilidade geral de leitura de radiografias de tórax.

Palavras-chave: Tuberculose pulmonar; Radiologia; Educação médica.

Introduction

Tuberculosis (TB) is a major health problem in Brazil. The TB incidence rate in the state of Rio de Janeiro is one of the highest in the country.(1)
In a large number of patients with respiratory symptoms, the presumptive diagnosis of TB is based on symptoms and abnormalities on chest X-rays.(2) Therefore, being able to interpret chest X-rays is an important and attainable skill, and its instruction should be pursued by medical institutions.

For many years, organizations and institutions in the United States and in the United Kingdom have assessed the issues on medical curricula related with teaching the interpretation of X-rays.(3-12) In addition, with the worldwide challenge posed by TB, the issue of the ­interpretation of chest X-rays for the diagnosis of TB reappears in national programs for TB control.(13)

In Brazil, the TB challenge has yet to be met, and, to our knowledge, neither physicians nor medical students have been surveyed on their chest X-ray interpretation skills. The objective of the present study was to evaluate senior medical students who have received formal education on the interpretation of chest X-rays and to determine their competence in diagnosing TB based on their reading of chest X-rays, as well as to identify factors associated with high scores for the overall interpretation of chest X-rays.

Methods

In October of 2008, we recruited a convenience sample of senior medical students who had received formal training in radiology at the Federal University of Rio de Janeiro Medical School, in the city of Rio de Janeiro, Brazil. The study was conducted at the Federal University of Rio de Janeiro Clementino Fraga Filho University Hospital, also in the city of Rio de Janeiro. The research ethics committee of the institution approved the study, and all of the participants gave written informed consent.

Selection of chest X-rays

From among 200 chest X-rays of patients with respiratory symptoms who had sought assistance at a publicly funded primary-care clinic, a case set of 6 was selected by three radiologists specializing in chest radiology. Although an actual clinical history was provided for each chest X-ray,(14,15) the radiologists were blinded to the final diagnoses. In 3 of the 6 cases selected, TB was confirmed by microbiological testing, whereas it was ruled out in the remaining cases. The non-TB cases presented with respiratory symptoms commonly seen at primary care clinics. The confirmed TB cases represented a spectrum of the disease, from minimal to extensive (Figures 1a, 1b and 1c). In two of the comparative cases, the chest X-rays were normal, one being of an overweight patient (Figures 2a and 2c). The remaining comparative case was a case of bronchiectasis that was confirmed with a CT scan (Figure 2b). The chest X-ray findings were classified according to the American Thoracic Society standards.(16)
In Brazil, medical schools share a core curriculum without specific instruction in radiology. A medical undergraduate course takes six years, which are organized into semesters.








Selection of medical students and teaching hours

The study population consisted of a convenience sample of 60 senior medical students on rotation in the Department of Internal Medicine (DIM), one and a half years before they applied to the national residence programs. All of the medical students had undergone a mandatory formal training course in radiology during the fourth (ten hours of chest radiology) and fifth (twelve hours of chest radiology) semesters. In the sixth semester, they received an eight-hour training course on TB diagnosis only (lectures and discussion of clinical TB cases). The students were also expected to have completed emergency rotational training, including off-campus experience.

Competence evaluation

During the study period, one of the authors was responsible for the application of the test to the medical students, in small groups. The medical students initially completed a questionnaire regarding their age, gender, career interest, years of emergency training and year of study. The participants were then presented with each of the 6 chest X-rays, one at a time, with a time limit of 4 min to interpret each image, and were asked to choose among three possible interpretations: normal image, probable diagnosis of TB and probable diagnosis of another pulmonary abnormality. In an attempt to evaluate coherence for a given chest X-ray interpretation, the medical students were also asked to choose among four possibilities for the subsequent clinical approach: discharge with counseling; request for a sputum smear test; prescription of a course of antibiotics (not specific for TB); and request for a new chest X-ray or other diagnostic tests. The medical students were expected to request a sputum smear test for a coherent subsequent approach to a suspected case of TB. Regarding non-TB cases, we considered it acceptable to discharge the patient with a previous common cold and dry cough with a normal chest X-ray. In the case of the patient with bronchiectasis, we considered it acceptable to prescribe antibiotics or to continue the diagnostic investigation, and we considered it appropriate to continue the diagnostic investigation in the case of the overweight patient with respiratory symptoms and a normal chest X-ray.

Data analysis

The distribution of the choices made by the medical students regarding the individual chest X-rays was evaluated. In addition, the proportions of their choices toward an appropriate clinical approach based on the history and the chest X-ray of each patient were computed. The group was also split into high scorers (5-6 correct answers) and low scorers (all other scores) in an attempt to determine the factors that could be associated with a higher score in the interpretation of chest X-rays, using Pearson's chi-square test. The sensitivity and specificity of the performance indexes were calculated considering the three TB confirmed cases as positive cases and the other three pulmonary conditions as negative cases.

Values of p < 0.05 were considered statistically significant. Analyses were performed using the Statistical Package for the Social Sciences, version 13.0 (SPSS Inc., Chicago, IL, USA).

Results

Eight students were excluded for providing incomplete answers on the questionnaire. Therefore, the final sample comprised 52 students. The median age was 24 years, and the sample was relatively homogeneous in terms of the future residence program (DIM, other) and time spent in emergency training. The gender distribution was nearly equal. Over half of the medical students were sixth-year students on DIM rotation. The only factor associated with a higher score for the overall interpretation of chest X-rays was the year of study (Table 1).




As shown in Table 2, the proportion of correct diagnoses of TB based on the chest X-rays was high. Sensitivity was, respectively, 86.5%, 90.4%, and 94.2% according to the severity of the disease (minimal, moderate and extensive). The coherence between the correct interpretation of the chest X-rays of TB patients and a suitable clinical approach was 100% (minimal and moderate) and 91.8% (extensive).



Very few medical students were able to interpret the chest X-ray of the overweight patient (5.8%). Most considered it a probable case of TB (false-positive), which lowered the specificity. However, in the interpretation of the other two non-TB chest X-rays (normal and bronchiectasis), the performance improved, with a specificity of 90.4% and 82.7%, respectively. The coherence between the interpretation of the non-TB chest X-rays and a suitable clinical approach was 71.4%, 89.1% and 0%, respectively, for the (normal) chest X-ray of the non-overweight patient, the X-ray of the patient with bronchiectasis and the (normal) chest X-ray of the overweight patient.

Discussion

In the present study, the competence of senior medical students in interpreting chest X-rays showed a sensitivity that was higher than was its specificity. The medical students performed better when the TB was extensive than when it was moderate or minimal. Therefore, the sensitivity was lower when there was minimal TB, as would be expected when a disease spectrum is used in diagnostic tests.(17) Regarding the two normal chest X-rays, the sensitivity was considerably lower for the chest X-ray of the overweight patient. Earlier studies have shown that readers do not perform well when interpreting normal chest X-rays, providing false-positive readings mostly due to parenchymal densities.(6,12,18)

Accordingly, in our study, we found more false-positives than false-negatives. However, this finding is not in the same range as that reported in one study of the accuracy of chest X-ray interpretation among radiologists and residents.(19) The higher proportion of false-positives in our study might reflect the fact that the medical students, who were aware of the purpose of the study, might have considered abnormal parenchymal densities as a probable TB feature.

The coherence following the interpretation of the chest X-rays as representing suspected cases of TB was reasonable, probably due to the intensive TB education that was provided in this setting. However, the overall interpretation of chest X-rays and the subsequent clinical approach were disappointing.

In settings where radiological evaluation is not provided in real time, a longer interval between the evaluation of chest X-rays and the medical decision-making could hamper the entire diagnostic work-up. In Brazil, it could impair TB control. In this sense, formal training in chest X-ray interpretation, in addition to formal TB courses, is crucial.

Since all of the medical students received formal training in radiology as well as formal TB education during their first medical years, we found that
the only factor associated with higher scores in the interpretation of chest X-rays was the year of study. As demonstrated in earlier studies, our results suggest that training might play a role in improving the performance of medical students in interpreting chest X-rays.(5,7)

Recently, in a report about learning and teaching activities among third-year medical students in the United States, the perceptions of the students regarding high-quality teaching were associated with learning how to interpret chest X-rays, among other factors.(4) In addition, a survey involving practicing physicians in the United States revealed that they believed that formal instruction in radiology should be mandatory in medical schools.(20)

In Brazil, unlike in countries with higher income, radiology training is not mandatory in undergraduate medical courses. According to the Brazilian National Accreditation System for Undergraduate Medical Schools, the curriculum guidelines, in its fifth and sixth articles, emphasizes that: ". . . medical students, prior to graduation, must demonstrate competence in history taking, physical examination (. . .) evidence-based prognosis, diagnosis and treatment of diseases". The lack of the specific nomination of diagnostic procedures gives rise to the enormous variety of curricula offering less than what is required.

Our study has several limitations. Because senior medical students were invited to take part in this study, those who were more comfortable with diagnosing TB or interpreting chest X-rays would be more likely to self-select for the study and consequently inflate the proportion of correct answers. In addition, the power was not enough to discriminate other possible factors associated with the high scores. However, we did not use the teaching files for chest X-ray sampling, and, by doing so, we guaranteed our sample of chest X-rays to be unknown to the students.

Regarding the instrument used to discriminate interpretation skills, the multiple choice approach was chosen for operational reasons.

Unfortunately, it has not been validated and it certainly represents a methodological weakness. Written descriptions of images have more support from earlier studies, although they also lack validity.(5-7)

Due to the purposely arranged bias related to the spectrum and the context, our estimates cannot be generalized to chest X-rays obtained from the general population treated at primary care clinics. The context bias could have inflated false-positive identifications of TB cases.(17,21) A wider sampling of chest X-rays, representing a more reliable TB prevalence, could be of help in future studies.

To our knowledge, this is the first time that medical students in Brazil have been evaluated in terms of their competence in interpreting chest X-rays. This study could represent the first step for implementing radiology, as well as TB diagnosis, as formal specialties in all medical schools in Brazil.

In conclusion, the competence in interpreting chest X-rays of TB patients was high among senior medical students who had received formal training in radiology and TB in their first years of medical school. The year of study was the only factor associated with a high score for the overall interpretation of chest X-rays.

Although undergraduate medical curricula vary widely in Brazil, our study provides preliminary data regarding the possible benefits of formal training in TB and of teaching chest X-ray interpretation in a country with a high incidence of TB.

Acknowledgments

We thank Dr. Carlos H F Castelpoggy, Head of the Department of Internal Medicine. We are also indebted to the undergraduate medical students Marcus V. B. Bueno and Joubert B. A. Junior.

References

1. World Health Organization [homepage on the Internet]. Geneva: World Health Organization; c2008 [cited 2008 Oct 14]. WHO Report 2008 - Global tuberculosis control: Annex 1 - profiles of high-burden countries. Available from: http://www.who.int/tb/publications/global_report/2008/annex_1_download/en/index.html

2. Gordin FM, Slutkin G, Schecter G, Goodman PC, Hopewell PC. Presumptive diagnosis and treatment of pulmonary tuberculosis based on radiographic findings. Am Rev Respir Dis. 1989;139(5):1090-3.

3. O'Brien KE, Cannarozzi ML, Torre DM, Mechaber AJ, Durning SJ. Training and assessment of CXR/basic radiology interpretation skills: results from the 2005 CDIM Survey. Teach Learn Med. 2008;20(2):157-62.

4. Torre DM, Simpson D, Sebastian JL, Elnicki DM. Learning/feedback activities and high-quality teaching: perceptions of third-year medical students during an inpatient rotation. Acad Med. 2005;80(10):950-4.

5. Dawes TJ, Vowler SL, Allen CM, Dixon AK. Training improves medical student performance in image interpretation. Br J Radiol. 2004;77(921):775-6.

6. Jeffrey DR, Goddard PR, Callaway MP, Greenwood R. Chest radiograph interpretation by medical students. Clin Radiol. 2003;58(6):478-81.

7. Scheiner JD, Noto RB, McCarten KM. Importance of radiology clerkships in teaching medical students life-threatening abnormalities on conventional chest radiographs. Acad Radiol. 2002;9(2):217-20.

8. Samuel S, Shaffer K. Profile of medical student teaching in radiology: teaching methods, staff participation, and rewards. Acad Radiol. 2000;7(10):868-74.

9. Young M, Marrie TJ. Interobserver variability in the interpretation of chest roentgenograms of patients with possible pneumonia. Arch Intern Med. 1994;154(23):2729-32.

10. Kuritzky L, Haddy RI, Curry RW Sr. Interpretation of chest roentgenograms by primary care physicians. South Med J. 1987;80(11):1347-51.

11. Kaufman B, Dhar P, O'Neill DK, Leitman B, Fermon CM, Wahlander SB, et al. Chest radiograph interpretation skills of anesthesiologists. J Cardiothorac Vasc Anesth. 2001;15(6):680-3.

12. Eisen LA, Berger JS, Hegde A, Schneider RF. Competency in chest radiography. A comparison of medical students, residents, and fellows. J Gen Intern Med. 2006;21(5):460-5.

13. Graham S, Das GK, Hidvegi RJ, Hanson R, Kosiuk J, Al ZK, et al. Chest radiograph abnormalities associated with tuberculosis: reproducibility and yield of active cases. Int J Tuberc Lung Dis. 2002;6(2):137-42.

14. Schreiber MH. The clinical history as a factor in roentgenogram interpretation. JAMA. 1963;185:399-401.

15. Loy CT, Irwig L. Accuracy of diagnostic tests read with and without clinical information: a systematic review. JAMA. 2004;292(13):1602-9.

16. Diagnostic Standards and Classification of Tuberculosis in Adults and Children. This official statement of the American Thoracic Society and the Centers for Disease Control and Prevention was adopted by the ATS Board of Directors, July 1999. This statement was endorsed by the Council of the Infectious Disease Society of America, September 1999. Am J Respir Crit Care Med. 2000;161(4 Pt 1):1376-95.

17. Ransohoff DF, Feinstein AR. Problems of spectrum and bias in evaluating the efficacy of diagnostic tests. N Engl J Med. 1978;299(17):926-30.

18. Herman PG, Gerson DE, Hessel SJ, Mayer BS, Watnick M, Blesser B, et al. Disagreements in chest roentgen interpretation. Chest. 1975;68(3):278-82.

19. Eng J, Mysko WK, Weller GE, Renard R, Gitlin JN, Bluemke DA, et al. Interpretation of Emergency Department radiographs: a comparison of emergency medicine physicians with radiologists, residents with faculty, and film with digital display. AJR Am J Roentgenol. 2000;175(5):1233-8.

20. du Cret RP, Weinberg EJ, Sellers TA, Seybolt LM, Kuni CC, Thompson WM. Role of radiology in medical education: perspective of nonradiologists. Acad Radiol. 1994;1(1):70-4.

21. Egglin TK, Feinstein AR. Context bias. A problem in diagnostic radiology. JAMA. 1996;276(21):1752-5.



Study carried out at the Federal University of Rio de Janeiro Medical School, Rio de Janeiro, Brazil.
Correspondence to: Vânia Silva. Rua Santa Amélia, 88/A/803, Tijuca, CEP 20260-030, Rio de Janeiro, RJ, Brasil.
Tel 55 21 9946-1027. E-mail: vmcsilva@fastem.com
Financial support: This study was funded in part by a grant from the Fundação de Amparo a Pesquisa do Estado do Rio de Janeiro (FAPERJ, Foundation for the Support of Research in the State of Rio de Janeiro; grant no. E-26/170.026/04).
Submitted: 14 August 2009. Accepted, after review: 27 October 2009.
** A versão completa em português deste artigo está disponível em www.jornaldepneumologia.com.br


Sobre os autores

Vania Maria Carneiro da Silva
Adjunct Professor. Department of Internal Medicine, Federal University of Rio de Janeiro Medical School, Rio de Janeiro, Brazil.

Ronir Raggio Luiz
Associate Professor. Department of Biostatistics, Federal University of Rio de Janeiro Medical School, Rio de Janeiro, Brazil.

Míriam Menna Barreto
Radiologist. Federal University of Rio de Janeiro Clementino Fraga Filho University Hospital, Rio de Janeiro, Brazil.

Rosana Souza Rodrigues
Radiologist. Federal University of Rio de Janeiro Clementino Fraga Filho University Hospital, Rio de Janeiro, Brazil.

Edson Marchiori
Full Professor of Radiology. Fluminense Federal University Medical School, Niterói, Brazil.