Poor Reproducibility in the Evaluation of Paranasal Sinus X-Rays in Chronic Rhinosinusitis
A. Luukkainen1,2*, E. Terna1*, J. Numminen3, A. Markkola4, P. Dastidar5, J. Jarnstedt5, H. Huhtala6, M. Karjalainen3, K. Blomgren7, P. Kauppi8, M. Rautiainen3,9, S. Toppila-Salmi1,8
1Transplantation Laboratory, Haartman Institute, University of Helsinki, Helsinki, Finland.
2Department of Otolaryngology, Yong Loo Lin School of Medicine, National University of Singapore, Singapore.
3Department of Ear and Oral Diseases, Tampere University Hospital, Tampere, Finland.
4University of Helsinki and HUS Imaging, Helsinki, Finland.
5Medical Imaging Centre, Department of Radiology, Tampere University Hospital, Tampere, Finland.
6School of Health Sciences, University of Tampere, Tampere, Finland.
7Department of Otorhinolaryngology, University of Helsinki and Helsinki University Hospital, Helsinki, Finland.
8Department of Allergy, University of Helsinki and Helsinki University Hospital, Helsinki, Finland.
9Department of Otorhinolaryngology, University of Tampere, Tampere, Finland.
DOI: 10.4236/ojrad.2017.71003   PDF   HTML   XML   1,460 Downloads   2,250 Views   Citations


Objective: The aim of this study was to evaluate intra- and inter-observer reproducibility of sinus x-rays in comparison to sinus computed tomography (CT) in chronic rhinosinusitis (CRS) patients. Methods: This was a prospective controlled study for which 14 adult CRS patients were recruited. Patients underwent a sinus multi-detector CT scan as well as additional sinus x-rays at the same time. Symptom interview and skin prick tests were performed. Lund-Mackay (LM) scores and 43 other findings in paranasal sinuses were analyzed by three blinded observers from CT-scans and x-rays. We compared agreement between sinus CT and x-rays (intra-observer reproducibility) and between three observers (inter-observer reproducibility) by Cohen’s kappa. Results: In at least 90% of the cases, the status of 47/49 structures was detectable in CT scans, whereas the status of only 8/49 structures was detectable in x-rays. The majority of the 25 visualized structures had poor intra-observer and inter-observer reproducibility. Conclusion: Only a few structures can be visualized in paranasal sinus x-rays and compared to paranasal sinus CT-scans, their reproducibility is poor. Our results strongly support the current consensus of radiation dose reduction by limiting the number of x-rays.

Share and Cite:

Luukkainen, A. , Terna, E. , Numminen, J. , Markkola, A. , Dastidar, P. , Jarnstedt, J. , Huhtala, H. , Karjalainen, M. , Blomgren, K. , Kauppi, P. , Rautiainen, M. and Toppila-Salmi, S. (2017) Poor Reproducibility in the Evaluation of Paranasal Sinus X-Rays in Chronic Rhinosinusitis. Open Journal of Radiology, 7, 23-34. doi: 10.4236/ojrad.2017.71003.

1. Introduction

Chronic rhinosinusitis (CRS) is a multifactorial and variable disease with a prevalence of 10.9% [1] . Computed tomography (CT) scans and/or nasal endoscopy are the recommended imaging modality for CRS [1] . Correlation between sinonasal symptoms and endoscopic or radiologic signs is poor [2] . The main findings of CRS are mucosal changes within the ostiomeatal complex and/or sinuses [1] . Paranasal sinus anatomical variants are very common and several critical anatomical structures (such as big vessels, orbit and central nervous system) are closely located to the sinonasal surgical area [3] [4] .

CRS surgery has improved many patients’ quality of life with hard to treat CRS [1] . Imaging of the nose and paranasal sinuses has progressed rapidly during the past decade. CT can demonstrate sinus anatomy specifically as well as anatomical variants and other important structures, when preparing for surgery or evaluating the cause of long lasting sinus symptoms. X-rays are not recommended for CRS imaging [2] . A simple radiogram of the nose and paranasal sinuses carries a radiation dose of 0.03 mSv (four days of natural background radiation). Although x-rays are inferior to CT in detecting bony structures and mucosal changes, little comparative data exists on the agreement between sinus CT and x-rays [2] .

Despite the novel low dose sinus CT scan modalities, the number of performed sinus x-rays is high in Finland (13.1 per 1000 inhabitants in 2011, according to the statistics of the Radiation and Nuclear Safety Authority) [5] . The aim of this prospective controlled study was to evaluate intra- and inter-observer reproducibility of sinus x-rays in comparison to sinus CT, in order to contribute to the need for reduction of unnecessary radiation.

2. Materials and Methods

2.1. Ethical Consideration

The study was approved by the ethics committee of the Pirkanmaa Hospital District (no 96032) and was conducted in accordance with the Helsinki Declaration of 1975, as revised in 1983. Written informed consent was obtained from each participant. Volunteer patients were exposed to an extra radiation dose of 0.02 years (6 days of natural background radiation in Finland).

2.2. Patients

This study was carried out in the Department of Otorhinolaryngology, at Tampere University Hospital, Finland from 2006 to 2015. A random sample of 14 adult CRS patients, who also had a history of AR and requiring sinus CT scans during 2007-2011, were enrolled. 3 (21%) out of 14 patients also had concomitant asthma. Having another severe disease was an exclusion criterion. 2 (85.7%) out of 14 patients reported as having other diseases (one had arrhythmia, one had resolved melanoma). None of the patients reported using regularly other medication than those due to inflammatory airway diseases. There was no patient record report of any other diseases, nor of psychiatric/psychologic disorders. Patient data was collected from medical records and by a questionnaire at the time of sinus CT scans, as previously described [6] . Follow-up data was collected from patient records of the Tampere University Hospital or Tampere City Hospital in 2015. The median (min-max) follow up time was 6.0 (0 - 8) years after sinus CT scans were performed. None of the subjects had undergone aspirin desensitization, allergen immunotherapy or anti IgE therapy prior to or during sinus CT scans or during follow-up.

2.3. CT Scans

Patients underwent routine sinus multiple detector CT scans for clinical purposes. Two different CT scanners were used: GE LightSpeed 16 (GE Healthcare, Milwaukee, Wisconsin) and Philips Brilliance 64 (Philips, Best, Netherlands). Patients were scanned in a supine position with a kilovoltage of 120 kV and a milliampere second of 100 mAs. With the GE scanner, slice thickness was 0.625 mm with coronal reconstructions at 1.5 mm and a radiation dose of 0.8 mSv. With the Philips scanner, slice thickness was 0.9 mm with coronal reconstructions at 0.9 mm and a radiation dose of 0.9 mSv. Both were three dimensional (3D) in nature without any gaps. In all cases, imaging was performed using a bone filter technique. Scans covered the entire sinonasal area in both axial and coronal directions, starting from the nasal tip and ending at the posterior wall of the sphenoid sinuses.

2.4. X-Rays

X-rays were performed at the same time as sinus CT scans. Two X-ray projections, Waters and PA-projection (Caldwell), were taken with Philips Pendo Diagnost (Philips, Best, Netherlands) skull unit in a sitting position. Imaging parameters were 85 kV, 12 mAs with Waters, and in PA-projection 85 kV and 8 mAs. The images were captured using photostimulable phosphor plates, pixel size 0.1 mm and read with AGFA CR 25 CR (Agfa-Gevaert N. V. Mortsel, Belgium) system. The radiation dose of x-rays with two projections was 0.06 mSv.

2.5. Evaluation of CT Scans and X-Rays

CT scans and x-rays were observed by three independent observers blinded to each other and to patient history data: an experienced head and neck radiologist (AM), an experienced Ear Nose Throat (ENT)-and rhinosurgeon (JN), and a fifth year ENT resident (ST-S). Examination of the same patient’s images took place at least a week apart. The three observers filled a 49-item form of sinonasal structures from both CT-scan and x-ray bilaterally for each patient (Table 3). Each structure listed in the form could be scored by 2 - 5 different listed choices. Before evaluation of the CT scans, all choices were discussed between observers. Before this study, the observers had participated a pilot study with 15 CT scans [6] . The radiologist did not respond to the question: “Need for septoplasty”.

2.6. Data Analysis

Statistical analysis was carried out by SPSS Base 15.0 Statistical Software Package (SPSS Inc., Chicago, IL, USA). Cohen’s kappa was used to compare the degree of agreement between CT scans and x-rays (e.g. intra-observer agreement); and the inter-observer agreement of x-rays. The calculation is based on the difference between how much agreement is actually present compared to how much agreement would be expected to be present by chance alone. The established interpretation of Kappa-value is classified into 6 subgroups: Poor < 0.2, Fair 0.21 - 0.4, Moderate 0.41 - 0.6, Good 0.61 - 0.8 and Very Good 0.81 - 1.0. A value under zero means that the agreement is worse than that by chance [7] . Two-tailed P-values of <0.05 were considered statistically significant.

3. Results

3.1. Patient Characteristics

Patient characteristics are shown in Table 1. The median (min-max) age was 36.9 (20.0 - 54.3) years. 57.1% of patients underwent sinonasal operation within a year after the CT scans (Table 1). 2 (14.3%) of the patients reported suffering from diseases besides CRS ± AR ± asthma with regular need for medication. One of the patient reported having melanoma and the other patient cardiac arrhythmia.

3.2. Visualized Structures in X-Rays

The number of visualized structures in x-rays was 25, e.g. the structures that were visualized in at least one patient´s paranasal sinus x-rays, whereas the number of visualized structures in at least one patient’s CT scan was 49 (Table 2). The visualized 25/49 structures in x-rays were Lund-Mackay (LM)-scores (except ostiomeatal unit), mucosa and size of paranasal sinuses, mucosa of inferior and middle turbinates, nasal mucosal oedema and septal abnormalities (Table 2). Yet, reliability to even detect these structures in most cases of x-rays was poor (Table 2). In at least 90% of the cases, the status of 47/49 structures was detectable in CT scans, whereas for x-rays it was only 8/49 (Table 2). Several diagnostically and surgically important structures were not visible in x-rays, such as ostiomeatal complex, insertion of the uncinate process, lamina papyrachea and anterior ethmoid artery (Table 2).

3.3. Intra-Observer Agreement

We compared the degree of agreement between x-rays and CT scans of the 25 structures that were visualized in x-rays. In general, the intra-observer agree-

Table 1. Characteristics of the patients.

Abbreviations: SPT = skin prick test; AERD = patient-reported aspirin exacerbated respiratory disease; CT = computed tomography; VAS= visual analogue scale (0 - 10). 1At least one peroral corticosteroid treatment during the last 12 months. 2Self-reported and patient-record information.

Table 2. The list of visualized or non-visualized sinonasal structures of paranasal sinus x-rays.

The CT scans and x-rays were taken from 14 patients with chronic rhinosinusitis symptoms. Each patient underwent CT scans and x-rays at the same time. The columns show in alphabetical order the evaluated 49 structures from paranasal sinus CT-scans and x-rays. Visualized in x-rays = the 25 structures that were visualized in at least one patient’s paranasal sinus x-rays (left column); Non-visualized in x-rays = the 24 structures that were non-visualized in paranasal sinus x-rays of all cases (right column); OMC = Ostiomeatal complex; Not detectable = the percentage of the observer´s responses “The status of the structure is not detectable” of both sides. 1Evaluated by the ENT surgeon. Other structures evaluated by the radiologist.

ment was poor (kappa < 0.2) in the majority of structures, such as LM scores. Moderate and good agreement was only achieved for gross anatomical structures on the right hand-side only, concerning respectively, nasal septum deviation and size of the frontal sinus (Table 3). Fair or poor agreement was observed for the rest of the structures.

3.4. Inter-Observer Agreement

The 25 structures that were visualized in x-rays were evaluated by a radiologist, an ENT surgeon and an ENT resident. The inter-observer agreement between radiologist and ENT resident for x-rays was poor (kappa ≤ 0.02) in 88% of the structures and fair (kappa 0.21 - 0.4) for the rest (12%) of the structures. The agreement between radiologist and ENT surgeon for x-rays was poor or fair in 80% of the structures and the agreement between ENT surgeon and ENT resident was poor or fair in 92% of the structures.

4. Discussion

This study was carried out to evaluate intra- and inter-observer reproducibility of sinus x-rays in comparison to sinus CT scans. When this study was started, multi- detector CT scans had a high radiation dose (in average 0.9 mSv) and x-rays were still relative widely used due to a clearly smaller radiation dose (in average 0.03 mSV per image). After this, low-dose CT scans (such as cone beam CT scans) have emerged (radiation dose between 0.08 - 0.27 mSv) and hence have largely replaced both sinus x-rays and high-dose sinus CT scans [8] . The number of performed sinus x-rays is still high in Finland [5] . Consequently, this study could contribute to the reduction of unnecessary radiation.

Our main finding was that a small proportion of structures can be visualized in x-rays; and x-ray evaluations have poor reproducibility. CRS specific changes that should be observed in CT-scans including degree of opacification of the paranasal sinuses and/or obstruction of the ostiomeatal complex cannot be visualized or only poorly in simple x-rays, making CRS diagnosis unreliable [1] . Similarly, others found in a small study that engorged turbinates and opaque nasal fossa could be observed from paranasal sinus x-rays. In the x-rays of 19.7% of patients with nasal and/or paranasal symptoms during at least 8 weeks, no changes were observable [9] .

Intra-observer agreement was fair to poor regarding most of the 25 structures that could be visualized in plain x-rays. Similarly to us, others report in a study with 47 patients with acute rhinosinusitis that plain sinus radiograms have a low sensitivity for detecting sinus inflammatory changes in other paranasal sinuses besides the maxillary sinus, in comparison to CT-scans [10] .

Inter-observer agreement was poor for the majority of structures that could be visualized by the radiologist, ENT surgeon and ENT resident. Very good inter- observer agreement was only achieved in regard to structures that could not be visualized, between ENT resident and surgeon.

Table 3. Comparison of the degree of intra-observer agreement from paranasal sinus x-rays and computed tomography (CT)-scans.

The CT scans and x-rays were taken from 14 patients with chronic rhinosinusitis symptoms. Each patient underwent CT scans and x-rays at the same time. Agreement is presented only of the 25 structures that were detected in x-rays (Table 2). The 24 structures that were non-detectable in x-rays have been withdrawn from evaluation. The order of the structures is the same as they were in the evaluation form. Structures with substantial to almost perfect agreement level by Kappa-coefficient. 1Evaluated by the ENT surgeon. Other structures evaluated by the radiologist.

Simple X-rays of sinuses are currently used to exclude acute sinusitis. Long- term smoking, decline in lung function and poor health-related quality of life are risk factors for exacerbation of asthma exacerbations and emergency room [11] . Prevalence of chronic rhinosinusitis is twice as common in asthmatics needing acute care compared to those without emergency room. However, CRS is not an independent risk factor for acute care. With progressing severe symptoms not responding to antibiotics and other forms of treatment, especially in this patient group, simple X-rays are in some cases used to evaluate the signs and extent of acute bacterial infection [12] [13] .

The limitation of our study is the small sample size, which was due to ethical reasons and to difficulties in recruiting volunteer patients for additional x-ray images, carrying extra radiation. We acknowledge that small sample size hinders large scale extrapolation of results and selection bias may have occurred.

5. Conclusion

Only a very small number of structures can be visualized in paranasal sinus x-rays and their evaluation reproducibility is poor compared to paranasal sinus CT scans. Despite the small study sample size, our results strongly support the current European position paper on rhinosinusitis and nasal polyps consensus that sinus CT-scans are needed when estimating the need for surgical treatment of CRS (Figure 1).

Figure 1. 14 patients diagnosed with chronic rhinosinusitis, who underwent routinely multi-detector sinus computed tomography (CT) scans, were asked to take part in the study and to voluntarily undergo sinus x-rays in three projections at the time CT scans were taken. Three cases are presented in this panel. Coronal projection of CT scans of the three cases ((c), (f), (i)). Waters projection of the x-rays of the same cases ((a), (d), (g)); and Caldwell projection of the x-rays of the three cases ((b), (e), (h)). Lateral projections of the x-rays are not shown. Patient 1: thickened mucosa of the inferior wall of the maxillary sinuses and narrowly open ostiomeatal complexes are visualized in CT scans (c), whereas they are not visualized in the x-rays ((a)-(b)). Patient 2: thickened mucosa of the maxillary sinuses and anterior ethmoidal cells, air bubbles within the fluid of right maxillary sinus, obstructed ostiomeatal complex are visualized in CT scans (f), whereas thickened mucosa of the maxillary sinuses is only visualized in x-rays ((d)-(e)). Patient 3: thickened mucosa of the maxillary sinuses and anterior ethmoidal cells, and signs of previous middle mental antrostomies are visualized in CT scans (i), whereas thickened mucosa of the maxillary sinuses is poorly visualized in x-rays ((g)-(h)).


The authors thank MD Mikko Suvinen and MD Anna-Maija Kuukka for collaboration and research nurse Marja-Leena Oksanen for her excellent assistance.

Disclosure and Funding

The study was supported in part by research grants from the Ahokas Foundation, Competitive Research Funding of the Tampere University Hospital (Grants 9H067, 9J108, 9L087), the Finnish Cultural Foundation, the Finnish Association of Otorhinolaryngology and Head and Neck Surgery, the Finnish Medical Society Duodecim, the Finnish Society of Allergology and Immunology, Helsinki University Central Hospital Research Funds, the Tampere Tuberculosis Foundation, the Ida Montin Foundation, the Jane and Aatos Erkko Foundation, the Väinö and Laina Kivi Foundation and the Yrjö Jahnsson Foundation. The authors report no conflicts of interest.


*The authors contributed equally to this work.

Conflicts of Interest

The authors declare no conflicts of interest.


[1] Fokkens, W.J., Lund, V.J., Mullol, J., Bachert, C., Alobid, I., Baroody, F., et al. (2012) EPOS 2012: European Position Paper on Rhinosinusitis and Nasal Polyps 2012. A Summary for Otorhinolaryngologists. Rhinology, 50, 1-12.
[2] Holbrook, E.H., Brown, C.L., Lyden, E.R. and Leopold, D.A. (2005) Lack of Significant Correlation between Rhinosinusitis Symptoms and Specific Regions of Sinus Computer Tomography Scans. American Journal of Rhinology, 19, 382-387.
[3] Khojastepour, L., Mirhadi, S. and Mesbahi, S.A. (2015) Anatomical Variations of Ostiomeatal Complex in CBCT of Patients Seeking Rhinoplasty. Journal of Dentistry, 16, 42-48.
[4] Vogiatzi, T., Kloukos, D., Scarfe, W.C. and Bornstein, M.M. (2014) Incidence of Anatomical Variations and Disease of the Maxillary Sinuses as Identified by Cone Beam Computed Tomography: A Systematic Review. The International Journal of Oral & Maxillofacial Implants, 29, 1301-1314.
[5] Helasvuo, T. (2013) Number of Radiological Examinations in Finland in 2011. STUK-B 161, STUK, Helsinki.
[6] Karjalainen, M., Julkunen, A., Markkola, A., Dastidar, P., Huhtala, H., Suvinen, M., Kuukka, A.-M., Rautiainen, M., Numminen, J. and Toppila-Salmi, S. (2016) Reproducibility of 3 Mm-Slice-Thick Reconstruction of Paranasal Sinus Computed Tomography Scans. Open Journal of Radiology, 6, 39-48.
[7] Viera, A.J. and Garrett, J.M. (2005) Understanding Interobserver Agreement: The Kappa Statistic. Family Medicine, 37, 360-363.
[8] Al Abduwani, J., Zilin Skiene, L., Colley, S. and Ahmed, S. (2016) Cone Beam CT Paranasal Sinuses versus Standard Multidetector and Low Dose Multidetector CT Studies. American Journal of Otolaryngology, 37, 59-64.
[9] Da Lilly-Tariah, O.B. and Aniemeka, J. (2006) Plain Radiological Profile of Paranasal Sinuses in Chronic Nasal Diseases in University of Port Harcourt Teaching Hospital. Nigerian Journal of Medicine, 15, 305-308.
[10] Aalokken, T.M., Hagtvedt, T., Dalen, I. and Kolbenstvedt, A. (2003) Conventional Sinus Radiography Compared with CT in the Diagnosis of Acute Sinusitis. Dentomaxillofacial Radiology, 32, 60-62.
[11] Kauppi, P., Kupiainen, H., Lindqvist, A., Haahtela, T. and Laitinen, T. (2014) Long-Term Smoking Increases the Need for Acute Care among Asthma Patients: A Case Control Study. BMC Pulmonary Medicine, 14, 119.
[12] De Sutter, A., Spee, R., Peersman, W., De Meyere, M., Van Cauwenberge, P., Verstraete, K. and De Maeseneer, J. (2005) Study on the Reproducibility of the Waters’ Views of the Maxillary Sinuses. Rhinology, 43, 55-60.
[13] Rosenfeld, R.M., Piccirillo, J.F., Chandrasekhar, S.S., Brook, I., Ashok Kumar, K., Kramper, M., et al. (2015) Clinical Practice Guideline (Update): Adult Sinusitis. Otolaryngology-Head and Neck Surgery, 152, S1-S39.

Copyright © 2021 by authors and Scientific Research Publishing Inc.

Creative Commons License

This work and the related PDF file are licensed under a Creative Commons Attribution 4.0 International License.