ABSTRACT

Objective: To identify the diagnostic concordance in the interpretation of cervical smears. Material and Methods: Cross-sectional study from October 2011 to January 2013. 50 samples were read by every 4 cyto-technologists who are the total staffs that read and interpret cytological specimens from a public health institution of the city of San Luis Potosí, México. The cytological diagnosis was confirmed by a certified pathologist. Checklist was developed to determine the quality of the interpretation composed of three sections: adequate sample, inadequate sample and cytological diagnosis. The diagnostic report was made by Bethesda System 2001 classification. Diagnostic concordance was measured through the Kappa coefficient. To establish the differences in cytological diagnosis between each cyto-technologist, chi square test was applied. Results: The concordance is acceptable in the classification of samples as inadequate (k = 0.66). The negative diagnosis was a significant concordance between each cytotechnologist and pathologist; however, no case had very good concordance. In the epithelial abnormalities, only the cytotechnologist 2 had significant concordance with the pathologist, however, its concordance is low. In the case of glandular abnormalities, only cytotechnologist 1 had no significant concordance with the pathologist. The percentage of true negatives was 30%, true positives 20%, false negatives 50% and false positives 0 %. There are significant differences between the readings of cyto-technologist 1 and 2 (p < 0.008), between 1 to 4 (p < 0.001), between 2 and 3 (p < 0.05), and finally between 3 and 4 (p < 0.003). Conclusions: The concordance between cyto-technologists and pathologists is below the required minimum level set by national and international standards.

Keywords:

Papanicolaou Test, Cervical Cancer, Quality Control, Cytology

1. Introduction

Cervical cancer (CC) is a health priority not only for its high magnitude and importance, but because it is a preventable disease in almost 100%. In the world, this disease causes premature death of about 274,000 women each year; mainly affects the poorest and most vulnerable, with a negative effect on their families and their communities, so early lost heads of families, mothers, wives, sisters, grandmothers, all with an important and irreplaceable role. Statistics show that 83% of new cases and 85% of deaths from this cancer occur in developing countries, where is also the leading cause of cancer death in women [1] .

For 2009, the CC was the second most common cancer in women worldwide with 86% of cases and 88% of deaths in developing countries and is the leading cause of years of life lost (cancer) between women in Latin America and the Caribbean [2] . The use of conventional Pap smear has reduced mortality from cervical cancer in developed countries [3] when is applied in organized screening programs as when a large proportion of women screened in a timely manner [4] . This has not been the case in Latin America, where despite the availability of the Pap test, a lot of women die each year from CC. It has been shown that even with adequate quality control, conventional cytology has an average sensitivity of 53% (95% CI: 48.6% - 57.4%) in Europe and the United States [5] .

In México, the mortality has maintained a downward trend the past 15 years, with a rate of 25.3 deaths per 100,000 women aged 25 and older in 1990 and 14.6 in 2006, representing a decrease of 45%. However, this decline has not been uniform throughout the country; behavior similar to global way, in Mexico increased mortality is concentrated in states with lower human development index and rural areas, as in the case of the State of San Luis Potosi, which is a rate above the national average.

Among the problems discovered in the CC prevention program in Mexico is screening coverage 70.7% which is insufficient. Another major difficulty has to do with the quality of detection, which is determined by three indicators: the quality of the sample, the internal quality control and the rate of positivity. Observed factors affecting the quality of reading and interpreting it are: outdated or no maintenance equipment, supplies of low quality, areas of poor working and sub-optimal conditions, lack of compliance with the guidelines established in the manual cytology and pathology as the review of congruence colpo-cyto-histological, forms of training and quality control, as well as staff shortages according to the official standards of Mexico [6] .

One line of research is the training to health care providers, who are the link between the health care institution and patients, so it is critical to investigate what factors influencing a correct reading and interpretation of cervical cytology in cytology laboratories of an institution that serves of early detection of cervical cancer.

The results of this approach will support for designing interventions aimed at improving the quality of interpretation of cervical smears, as this may get more accurate diagnoses and together with other actions will likely contribute to the decreased mortality from this disease.

2. Material and Methods

It is a cross-sectional design. It was held in October 2011 to January 2013 was conducted in cytology laboratories in a public health institution of the city of San Luis Potosi (SLP), México. A kit of 50 lamellae from Pathology Department from Dr. Ignacio Morones Prieto Hospital were read by every 4 cyto-technologists who are total staff read and interpret cervical cytology samples in the capital city belonging to the organization of the study were evaluated. A checklist was developed to determine the quality of interpretation, which was integrated by three sections: adequate sample, inadequate sample, and cytological diagnosis. The diagnostic report was conducted by classifying the Bethesda 2001 System [7] (Table 1 and Table 2).

The diagnostic concordance was measured through the Kappa coefficient index validated by De Landis [8] (Table 3). To establish the differences in the cytological diagnosis successful and erroneous issued between cyto-technologists was applied chi square test.

Source: Solomon, D., Davey, D., Kurman, R., Moriarty, A., O'Connor, D., Prey, M. & Young, N. (2002) The 2001 Bethesda System: terminology for reporting results of cervical cytology. JAMA, 287(16), 2114-2119.

Source: Solomon, D., Davey, D., Kurman, R., Moriarty, A., O’Connor, D., Prey, M. & Young, N. (2002) The 2001 Bethesda System: terminology for reporting results of cervical cytology. JAMA, 287(16), 2114-2119.

Source: De Landis, J. and Koch, G. (1977) The measurement of observer agreement for categorical data. Biometrics, 33, 159-174.

The validity of the instruments was performed by experts in the field. A certified pathologist served as gold standard, which validated the interpretation of the results issued by the cyto-technologists. The pilot test was conducted in July 2012 in a public health institution similar to the study.

50 sample of a kit already made by the team of pathologists from Central Hospital Dr. Ignacio Morones Prieto and obtained with a non-probabilistic method were selected. The number of platelets was determined based on the criteria for quality control assessments, established in the Official Mexican Standard 014SSA/1994 for the prevention, diagnosis, detection, treatment, control and surveillance of cervical cancer. Samples belong to different patients treated at the Central Hospital Dr. Ignacio Morones Prieto in 2010, of which no data are presented for not being the direct target of this study. For evaluation to each cyto-technologist was assigned a number of progressively from 1 to 4 and each read 50 sample.

The characteristics of the 50 selected specimens were: 2 inappropriate and 48 appropriate. The diagnoses were: 2 negative, 7 negative with inflammation, 4 negative with microorganisms, 32 epithelial abnormalities (19 for low-grade lesions, 9 injury high degrees and 4 carcinomas) and 5 with glandular abnormalities. The cyto-technologists delivery kit slides were made to give reading this material as they do in their daily work, (day of 8 hours). Subsequently re-evaluated the pathologist's report read each of the cyto-technologists.

To carry out this research took into account the legal provisions on Bioethics pointing the Health Act, 2011 and the declaration of Helsinki, 2000. This project was submitted to the Ethics Committee of the Faculty of Nursing the Autonomous University of San Luis Potosí with GVI registration number 08-2011 and the Committee of Research and Bioethics at the institution where the study was conducted with registration number R-2012-2402-38, both committees gave their approval. All participants gave their written consent. The identity of the participants and the institution are safeguarded.

3. Results

The average age was 39.5 years. Seniority in the institution was 0 - 12 years with a mean of 5; the reading experience lamellae 9.7 years on average found. The average day lamellae read by cyto-technologist was 47.5. 75% of cyto-technologists are men. 25% have a technical degree, half with unfinished degree, the rest (25%) with completed degree.

In reviewing the concordance as regards the report on the quality of the samples shows that there is an acceptable concordance (k = 0.66) to agree on the classification of inadequate samples.

With regard to the cytological diagnosis concordance between the pathologist and each cytotechnologist it was noted that the negative diagnosis was a significant concordance between each cytotechnologist and pathologist; however, in no case it was very good concordance, according to the kappa coefficient values.

Regarding epithelial abnormalities, only the cytotechnologist 2 had significant concordance with the pathologist, however, its consistency is low. In the case of glandular abnormalities, only cytotechnologist 1 had no significant concordance with the pathologist (Table 4).

Exist statistically significant differences between the proportions of successful and misdiagnosis issued by the cyto-technologists, thus proved that there are significant

*It was considered only cytological diagnosis: Negative, epithelial abnormalities and glandular abnormalities. **The number of concordant results relates to the resulting values on the diagonal of each 2 × 2 table that formed. Source: Checklist of diagnosis.

differences between the readings of cyto-technologist 1 and 2 (p < 0.008) were found between the 1 to 4 (p < 0.001), between 2 and 3 (p < 0.05), and finally between 3 and 4 (p < 0.003) (Table 5).

It was noted that further highlight the differences are the way to perform the procedure (the reading of samples), years of experience performing this technique and the number of lamellae read daily. It was also found that the percentage of true negatives was 30%, 20% true positives, false negatives 50%, and there was not reported false positive (Figure 1).

The level of concordance between the gold standard (certified pathologist) and all the cyto-technologists is bad (k = 0.19) (Poor). By analyzing the concordance between each of the cyto-technologists, is that 100% does not have the minimum acceptable concordance, when the expectation is to have a concordance above 0.80 (good concordance) (Figure 2).

Cyto-technologists, p*: chi square test. Source: Checklist of diagnosis.

4. Discussion

Most of the staff working in the laboratories of the institution has an age greater or close to 40 years old, however only 50% of the cyto-technicians have over 10 years of experience in dyeing, reading and interpreting cervical smears. The academic level is heterogeneous as there staff having from a medium-basic level to degree. Most of the staff is unprofessional, which shows that there is no clear specification or profile given position for reading and interpretation of smears, as there are staff with 0 - 12 years of institutional seniority, with different academic level and have the same status. In this study, we observed that this cyto-technologist, who had a high academic level, affected a staining procedure, reading and interpretation of smears properly.

Although it is necessary to deepen the analysis of this variable, it was observed that apparently the experience is an important factor to reading and interpretation of smears, as the cyto-technologist who had more than 15 years reviewing and interpreting smears performed adequately the procedure. The total cyto-technologists have not received training in the past year concerning aspects of regulation, staining, reading and interpretation of cervical cytology. This condition could be a factor behind the performance that is presented as a study in Cuba [9] as that technical training affected the improvement of cytological diagnosis with increased positivity, concordance in general and according to the type of injury, in the cytological diagnosis.

In this research the average lamellae read by cyto-technologist in one day was 47.5. This is below what is regulated and outlined in the Procedures Manual Staining and Interpretation of Cervical Cytology from Health Secretary México [6] that is 56 slides per day and regulated by the organization itself is 50 lamellae. On the other hand, the average time a cyto-technologist invested in reading was 2.8 minutes per slide, which is below the established; it must be from 5 to 8 [6] . This aspect can be one of the factors that no abnormalities are detected in the samples, without regarding to considering other aspects mentioned by McCrory [10] , such as obsolete equipment, poor quality microscopes, outdated reagents and sub-optimal working conditions, among others.

An aspect that is worth noting is that the laboratory evaluated cytology has no microscope co-observation which is used for joint review between the cyto-technologists and pathologists as noted regulations [11] , critical factor given the evidence presented to a US study by Cibas [12] , which have a 34% concordance between cyto-technologists and pathologists and after corrective actions such as changing a binocular microscope, discrepancies decreased to 18.3%.

The process of reading and interpretation is assessed using a checklist containing eight sections: initial registration and reception, slide preparation, staining, mounting, reading and interpretation, final registration, internal quality control and characteristics of the microscope. 75% of cyto-technologists improperly performed the procedure. More errors were observed in the preparation, staining, internal quality control and knowledge of the characteristics of the microscope.

According to the results of this study, 25% of the cyto-technologists acquired a kappa index of 0.30 poor agreement, something like this was reported in a study conducted in Colombia by Cendales [13] and Mexico by Lazcano [14] , where it was found that there was great variability between observers and revealed that 34% of the cyto-technologists had a null concordance, likewise factors that are related with high levels in the diagnostic concordanceare: a level high knowledge, performance adequately staining procedure, reading and interpretation, 15 years of experience interpreting at least 50 samples per day.

In this study, a high percentage of diagnoses that did not coincide with those issued by the pathologist was found. This percentage was presented in a range of 6% to 34%. It is clear that all these misdiagnoses were false negatives. These cyto-technologists found no abnormalities on cytology, which there were identified by the pathologist. The above is consistent with previous studies such as Spence [15] who performed a meta-analysis and found that in Europe there are a proportion of 11.3% false negatives; which disagrees with that found in Mexico by Regil [16] , and Ortiz [17] it has been observed that the rate of false negatives varies from 8.3% to 19%.

By making a detailed analysis of the frequency of errors in diagnosis issued by cyto- technologists, we found that the true negatives were 30%, the 20% true positives, false negatives and 50% false 0% positive; this contrasts with findings by Regil [16] at the National Institute of Cancerologíaof México, who searched the frequency of abnormalities diagnosed by cytology and found that the true negative cases were 13%, true positive 61%, false negative 19% and 7% false positives. The quality shown in the program Cervical Cancer Early Detection is urgent to attend at all institutional levels and in all sectors whether public or private.

Quality control provisions cytology laboratories were analyzed, none of them is carried out routinely the joint review between the pathologist and the cyto-technologist positive or uncertain. In addition, the pathologist only checks the 5% of the slides classified as negative by cyto-technologists, which differs from what is indicated in the Mexican Official Standard [18] , where mentioned that the pathologist must review all day 100% of positive, atypical or questionable slides jointly with cyto-technologist who read them and perform a random check of 10% of the negative slides. It should be mentioned that an adequate quality control may decrease the proportion of false negatives up to 5.7%, as demonstrated Michelow [19] who tested the re-evaluation of negative cervical smears as a form of quality control internal.

The impact of these findings can be divided into two aspects: the administrative and side effects to users. In the administrative field has been identified that each day the number of “not useful” (smears misdiagnosis) cervical cytology not give an accurate diagnosis, so cytology should be repeated within a period not greater rises a year, which doubled spending resources in a single detection is caused. Also the problem of false negatives is that the patient will be diagnosed until symptoms of CC appear; implying that the disease is detected in advanced stages, so the treatment is much more expensive, than if it had been detected at earlier stages.

The effects on the user are the most important, they have a risk when the health service lacks quality, that every day is less usual to be measured it in terms of evaluation of the care process patient, the above can sustain at Donabedian [20] who says that the degree of quality is the measure to the attention is able to achieve the most favorable balance between risks and benefits, as the quality and cost money are interrelated, so, when attention is excessive (e.g. a test or procedure that is not required) and harmful, is more expensive and lower quality results; when attention is excessive and harmless, it is more expensive; but without increasing the quality, that is wasteful because the patient also spends resources to assist the service either in time or in transit. Therefore, it is crucial to call upon managers of health institutions for these investigations give guidelines to the design and implementation of strategies aimed at developing systems for monitoring and control of the service provider rather than achieving goals and indicators numerically contribute little to ensuring the quality of health services.

In this study the concordance between cyto-technologists and pathologists is below the required minimum level set by national and international standards. It reflects the absolute necessity of training staff in reading cervical cytology in different ways that expose current aspects of regulations, procedures, cell morphology, among others; besides it is necessary to conduct periodic evaluations of the daily work of the cyto-tech- nologists, with the aim of identifying opportunities.

It is important to reinforce the quality controls that are in cytology laboratories, through supervision and monitoring of the department, the joint review between the pathologist and the cyto-technologist both positive and negative cases. In addition to these actions must be implement periodic external quality controls and should be outside the institution because they tend to be more objective and provide information that is very useful for decision-making.

One of the main limitations of this research was the number of subjects of the study, because very few cyto-technologists assigned this role in a public institution, only 4 or 10 people were interpreting the specimens from various municipalities in the state of San Luis Potosi, and it would be desirable to implement this analysis in other institutions at regional or national level in order to verify whether this result are similar than other. However the evidence of this study is fundamental to the evaluation of health services.

For future research, it is recommended to explore other factors that may be related to quality in the reading and interpretation of samples as those related to the structure, staff expertise and quality of the specimen.

Cite this paper

Gutiérrez-Enríquez, S.O., Chávez-Hernández, L., Terán-Figueroa, Y., Gaytán-Hernández, D., Oros-Ovalle, C., Gallegos-García, V., Diáz-Oviedo, A. and González-Acevedo, C.E. (2016) Concordance in the Interpretation of Cervical Cytology for the Early Diagnosis of Cervical Cancer. Open Journal of Obstetrics and Gynecology, 6, 714-724. http://dx.doi.org/10.4236/ojog.2016.612089

References