Quality Indicators of Cervical Cytopathology Tests in the Public Service in Minas Gerais, Brazil

Abstract

Objective The objective of this study is to assess the performance of cytopathology laboratories providing services to the Brazilian Unified Health System (Sistema Único de Saúde – SUS) in the State of Minas Gerais, Brazil.

Methods This descriptive study uses data obtained from the Cervical Cancer Information System from January to December 2012. Three quality indicators were analyzed to assess the quality of cervical cytopathology tests: positivity index, percentage of atypical squamous cells (ASCs) in abnormal tests, and percentage of tests compatible with high-grade squamous intraepithelial lesions (HSILs). Laboratories were classified according to their production scale in tests per year ≤ 5,000; from 5,001 to 10,000; from 10,001 to 15,000; and ≥ 15,001. Based on the collection of variables and the classification of laboratories according to production scale, we created and analyzed a database using Microsoft Office Excel 97–2003.

Results In the Brazilian state of Minas Gerais, 146 laboratories provided services to the SUS in 2012 by performing a total of 1,277,018 cervical cytopathology tests. Half of these laboratories had production scales ≤ 5,000 tests/year and accounted for 13.1% of all tests performed in the entire state; in turn, 13.7% of these laboratories presented production scales of > 15,001 tests/year and accounted for 49.2% of the total of tests performed in the entire state. The positivity indexes of most laboratories providing services to the SUS in 2012, regardless of production scale, were below or well below recommended limits. Of the 20 laboratories that performed more than 15,001 tests per year, only three presented percentages of tests compatible with HSILs above the lower limit recommended by the Brazilian Ministry of Health.

Conclusion The majority of laboratories providing services to the SUS in Minas Gerais presented quality indicators outside the range recommended by the Brazilian Ministry of Health.

Resumo

Objetivo Avaliar o desempenho dos laboratórios de citopatologia prestadores de serviço para o Sistema Único de Saúde (SUS) no estado de Minas Gerais, Brasil.

Métodos estudo descritivo com base nos dados obtidos do Sistema de Informação do Câncer do Colo do Útero no período de janeiro a dezembro de 2012. Para avaliação da qualidade dos exames citopatológicos do colo do útero, foram analisados três indicadores de qualidade: índice de positividade, percentual de ASC (células escamosas atípicas) entre os exames alterados e percentual de exames compatíveis com HSIL (Lesão intraepitelial escamosa de alto grau). Os laboratórios foram classificados conforme escala de produção em: ≤ 5.000; entre 5.001 e 10.000; entre 10.001 e 15.000 e ≥ 15.001. Após a coleta das variáveis e classificação dos laboratórios conforme escala de produção foi elaborado um banco de dados, o qual foi analisado por meio do programa Microsoft Office Excel 97-2003.

Resultados Em 2012, no estado de Minas Gerais, 146 laboratórios prestaram serviço para o SUS realizando um total de 1.277.018 exames citopatológicos do colo do útero. Metade desses laboratórios apresentou escala de produção ≤ 5.000 exames/ano, e efetuaram 13,1% do total de exames do estado, por outro lado 13,7% dos laboratórios apresentaram escala de produção > 15.001 exames/ano e efetuaram 49,2% do total de exames do estado. O índice de positividade apresentado pela maioria dos laboratórios que prestaram serviço para o SUS no ano de 2012, independente de sua escala de produção, foi abaixo ou muito abaixo do recomendado. Dentre os 20 laboratórios que efetuaram mais de 15.001 exames por ano, somente três apresentaram o percentual de exames compatíveis com HSIL acima do recomendado pelo Ministério da Saúde.

Conclusão A maioria dos laboratórios prestadores de serviço para o SUS no estado de Minas Gerais apresentou indicadores de qualidade fora dos parâmetros recomendados pelo Ministério da Saúde.

Keywords

quality control - Pap smear - cervical cancer - screening programs - continuous education

Palavras-chave

controle de qualidade - exame Papanicolau - câncer de colo do útero - programas de rastreamento - educação continuada

Introduction

Several countries have developed and successfully implemented cervical cancer screening programs based on Pap smear testing, as shown by the decrease in cervical cancer incidence and mortality rates.[1] [2] Although the Pap test is low cost and easy to execute, its performance has been debated because of its high rate of false-negative results, at ∼20%, which has been attributed to errors in microscopy analysis and inter- and intra-operator variabilities.[3] [4] [5]

The internal quality control (IQC) in laboratories are defined as the continuous monitoring of all activities involved in the laboratorial function, which include management of human resources, operating procedures, and quality control. For this reason, the development of methods are needed to ensure that a laboratory meets standard levels of accuracy and reliability in their laboratory report.[6] [7] In Brazil and other parts of the world, such as the United Kingdom, Europe, Australia, and Mexico, IQC methods are used to reduce false-negative rates in laboratory testing.[8] [9]

In 2012, the Brazilian National Cancer Institute (Instituto Nacional do Câncer - INCA/MS), together with Brazilian Scientific Societies, developed the Quality Management Manual for Cytopathology Laboratories with the objective of improving the reliability of cytopathology testing.[10] This manual addresses several IQC methods and provides quality indicators that allow constant monitoring of cytopathology test results. In addition, these indicators allow the assessment of overall and individual performance levels of laboratory personnel who process and analyze cytopathology tests. Some of these indicators are the positivity index (PI), the percentage of atypical squamous cells (ASCs) of undetermined significance in abnormal tests, and the percentage of tests compatible with high-grade squamous intraepithelial lesions (HSILs) in satisfactory tests.[10]

To emphasize the importance of quality in cytopathology testing, the Brazilian Ministry of Health enacted an ordinance that establishes the National Cytopathology Quality Control Program for Cervical Cancer Prevention (Qualificação Nacional em Citopatologia na Prevenção do Câncer do Colo do Útero).[11] Both the Manual and the Ministry Ordinance have defined standards and criteria required to measure the quality of cytopathology testing and to assess the performance of public and private laboratories providing services to the Brazilian Unified Health System (Sistema Único de Saúde – SUS).

Nonetheless, well-defined criteria and recommendations to improve the quality of cytopathology testing in Brazil are not enough to solve one of its main challenges, which is the underperformance of laboratories providing services to the SUS throughout the country whose quality indicators fall outside the limits recommended by the Brazilian Ministry of Health.[12] In light of this issue, our study assessed the performance of cytopathology laboratories providing services to the SUS in Minas Gerais.

Methods

This is a descriptive study performed using data we obtained from the Cervical Cancer Information System (Sistema de Informação do Câncer do Colo do Útero – SISCOLO), from January to December 2012, in Minas Gerais, Brazil. We obtained the data through pre-selected variables using Tabnet, a SISCOLO application, available at the SUS Computer Department (Departamento de Informática do SUS – DATASUS). These data are in the public domain with non-restricted access.[13] Given that the study is based on test results collected in 2012 that are publicly available via SISCOLO, authorization from the Research Ethics Committee was not required.

In this study, we used IQC indicators recommended by the Brazilian Ministry of Health to assess the quality of cervical cytopathology tests performed by laboratories providing services to the SUS.

The first indicator assessed was the Positivity Index (PI), which expresses the prevalence of cellular alterations and the sensitivity of the screening process for detecting atypias or cervical cancer precursor lesions. Low positivity may indicate that positive samples are not being identified, resulting in false negatives. The Brazilian ministry of health recommends a PI higher than 3%.[10] [14]

The second indicator used was ASC percentage in abnormal tests, which is assessed together with PI. An apparently adequate PI may contain a high percentage of tests compatible with ASCs, which would reflect cytologically ambiguous cases. ASCs should correspond to less than 60% of abnormal tests.[10] [11]

The third and last indicator used was the percentage of tests compatible with HSILs in satisfactory tests. HSILs are considered to be cervical cancer precursor lesions, that is, lesions that may effectively progress to cancer. Thus, detection of HSILs is the main objective of cervical cancer prevention.[10] [12] This indicator measures the capacity to detect precursor lesions, and recommendations indicate that HSILs should be equal to or greater than 0.4% of satisfactory tests.[10] [11]

To perform a critical assessment of laboratory quality, we classified these three indicators according to the guidelines of the Brazilian Ministry of Health and the National Institute of Cancer (Instituto Nacional de Câncer - INCA),[10] as seen in [Table 1].

Table 1 Classification of quality indicators according to Brazilian Ministry of Health recommendations

Quality indicators	Classification
Positivity index > 3, ASC/Abnormal<60, and HSIL > 0.4	Recommended
Positivity index > 3, ASC/Abnormal<60, and 0.2<HSIL<0.4	Intermediate
2<Positivity index<3, ASC/Abnormal<60, and HSIL > 0.2	Low
Positivity index<2, ASC/Abnormal > 60, and HSIL<0.2	Very Low

Abbreviations: ASC/Abnormal, Percentage of atypical squamous cells (ASCs) in abnormal tests; HSIL, Percentage of high-grade squamous intraepithelial lesions (HSILs) in satisfactory tests.

Data source: Brazil (2012).[10]

Regarding quality criteria, the Brazilian Ministry of Health Ordinance n. 3,388/2013[11] (QualiCito-Portaria) recommends that Type I Laboratories – defined as public or private laboratories that provide services to the SUS – processing cervical cytopathology tests should have a minimum production of 15,000 tests/year.[11] To compare the performance of laboratories according to their production scale, laboratories were classified as ≤ 5,000; from 5,001 to 10,000; from 10,001 to 15,000; and > 15,000 tests/year.

Based on the collection of variables and the classification of laboratories according to production scale, we created and analyzed a database using Microsoft Office Excel 97–2003.

Results

In 2012, 146 laboratories provided services to the Brazilian SUS in Minas Gerais and performed 1,277,018 cervical cytopathology tests. Half of these laboratories had production scales lower than or equal to 5,000 tests/year and accounted for 13.1% of all tests performed in Minas Gerais in 2012. A total of 36 laboratories, with production scales between 5,001 and 10,000 tests/year, performed 20.8% of all cervical cytopathology tests in Minas Gerais. Laboratories with production scales of 10,001 to 15,000 tests/year (17 laboratories in 2012) performed 16.9% of all tests. The remaining 13.7% of laboratories, with production scales greater than 15,000 tests/year, accounted for 49.2% of all tests performed in the entire state during 2012 ([Table 2]).

Table 2 Distribution of cytopathology laboratories serving the SUS in Minas Gerais according to production scale and processed tests in 2012

Production scale	Laboratories		Tests processed
	n	%	n	%
≤ 5,000 tests/year	73	50.0	167,443	13.1
5,001 to 10,000 tests/year	36	24.7	265,003	20.8
10,001 to 15,000 tests/year	17	11.6	215,440	16.9
> 15,000 tests/year	20	13.7	628,681	49.2
Total	146	100.0	1,277,018	100.0

Data source: SISCOLO/DATASUS.[13]

[Table 3] describes the classification of quality indicators according to the production scales of the 146 laboratories that provided services to the SUS in Minas Gerais during 2012. Of 73 laboratories performing up to 5,000 tests/year, only 11 showed indicators in the recommended range; the indicators of 47 of these laboratories, corresponding to more than half of the laboratories included in the group, were well below the recommended limits.

Table 3 Classification of quality indicators of laboratories serving the SUS in Minas Gerais according to production scale

Classification of indicators	Recommended		Intermediate		Low		Very low		Total
Production scale Tests/year	n	%	n	%	n	%	n	%	n	%
≤ 5,000	11	7.5	7	4.8	8	5.5	47	32.2	73	50.0
5,001 to 10,000	2	1.4	4	2.7	4	2.7	26	17.8	36	24.7
10,001 to 15,000	1	0.7	1	0.7	2	1.4	13	8.9	17	11.6
> 15,000	3	2.0	1	0.7	4	2.7	12	8.3	20	13.7
Total	17	11.6	13	8.9	18	12.3	98	67.2	146	100.0

Data source: SISCOLO/DATASUS.[13]

Regarding laboratories processing from 5,001 to 10,000 tests/year, 26 out of 36 laboratories of this group showed indicators that were much lower than recommended; only two laboratories had indicators in the recommended range. Of the 17 laboratories classified with production scales of 10,001 to 15,000 tests per year, only one laboratory presented quality indicators in the recommended range; 13 of these laboratories were well below the recommended values.

Finally, of the 20 laboratories that performed more than 15,000 tests per year, only three had quality indicators within the recommended range and in 12 of these laboratories, the indicators were well below recommended limits.

Thus, overall, 98 laboratories (67.2%) showed quality indicators well below recommended limits, and only 17 (11.6%) of the 146 laboratories serving the SUS in 2012 presented quality indicators within the range recommended by the Brazilian Ministry of Health.

Discussion

The objective of quality control in cytopathology is to improve the test's performance to eliminate false-negative results.[14] [15] From a practical standpoint, false-negatives are of greater concern in routine testing than false-positive results, given that non-diagnosed women may not seek follow-up and will live with the risk of developing precursor lesions.[16] In turn, false-positive results also have negative effects because they lead to unnecessary surgeries and clinical procedures that can negatively affect women's reproductive and sexual life, not to mention the clear psychological impacts of these procedures.[3]

The PI is an indicator that expresses the prevalence of cellular alterations in cytopathology tests and the screening program's sensitivity for detecting lesions in the examined population. Brazil has a low PI associated with a high number of ASC alterations, indicating that cervical cancer precursor lesions are not being diagnosed.[12]

The results of this study showed that most laboratories providing services to SUS in 2012, regardless of production scale, showed PI values well below the Ministry of Health's recommended range, which is a PI greater than 3%.[10] Even when laboratories showed intermediate quality indicators, with PIs greater than 3%, the detection levels of HSILs were low.

The distribution of cervical cytopathology tests among a large number of laboratories with small production scales may have a negative effect on the professional expertise of the personnel who analyze cytological slides, thus leading to false-negative results. Moreover, a production of less than 5,000 tests/year is also a possible cause of higher error rates during slide reading.[17] The Brazilian Ministry of Health, as stated in ordinance n. 3,388 of December 2013, encourages laboratories to perform more than 15,000 tests/year.[11] However, this study showed a greater proportion of laboratories with quality indicators within the recommended range for the ‘production scale ≤ 5,000 tests/year’ category.

A working group composed of cytologists and cytotechnologists was created to analyze data and to debate quality and good practices in gynecological cytopathology laboratories. This working group discussed the issue of laboratories with cytopathology test volumes below 5,000 tests/year. According to the results presented, it is unclear how these laboratories should be monitored because the literature has been based on various assumptions, which means that further research on this subject is needed.[18]

The low PI values found in this study – compared with the values observed in countries with successful cervical cancer prevention programs, such as the United States (PI 6.8%) and the United Kingdom (PI 6.4%) – indicate the presence of false-negative results and support the need to reevaluate and/or implement IQC in such laboratories.[10] [19]

Assessing the diagnostic frequency of ASCs may contribute in elaborating laboratory quality control measures, establishing national standards, and reaching acceptable percentages for ASC diagnosis. In addition, the constant monitoring of this diagnostic category enables the evaluation of its behavior over time and, if there are changes, it allows the opportunity to assess and explain them. High numbers of false-negatives and false-positives and an excess of ASC diagnoses in cervical cytopathology reports have raised global concerns. Thus, the adoption of IQC measures in laboratories that perform this type of test is crucial to minimize these problems.[20]

HSILs are considered to be the actual precursor lesions to cervical cancer, as these lesions can progress to cancer. This makes HSIL detection the main objective of cervical cytopathology tests. For this reason, the percentage of tests compatible with HSIL is an indicator of the capacity to detect precursor lesions and should be greater than 0.4%, a value based on results obtained in countries with successful screening programs able to reduce the incidence and mortality of cervical cancer, such as the United States (HSIL 0.5%).[21]

A concern raised by our study is that almost 80% of laboratories serving the SUS in 2012 have shown values for the indicator HSIL/Sat (high squamous intraepithelial lesion / satisfaction) below or well below the recommended levels, regardless of the production scale. Of 20 laboratories performing more than 15,000 tests/year – which are responsible for more than 50% of all tests performed in Minas Gerais – only three showed HSILs in the recommended range; 14 of these high-volume laboratories showed HSIL values below or well below the recommended, indicating that cervical cancer precursor lesions are not being detected. The performance of IQC in cytopathology laboratories aims to reduce misdiagnoses of high-grade lesions by retesting cytological slides, because a slide previously classified as negative might be identified as a false-negative.[22]

A group of researchers assessed quality indicators in laboratories that performed External Quality Control (EQC) and found that all quality indicators – particularly high-grade lesion detection – improved in these laboratories after continuous education. This result demonstrates the importance of EQC in improving the capacity of detecting cervical cancer precursor lesions in professionals that perform cytopathology tests.[23] The correct identification, diagnostic confirmation, treatment, and follow-up of these cellular alterations, which represent true cervical cancer precursor lesions, might prevent their progression to invasive cancer.[12] [24]

In Minas Gerais, cervical cancer is the cancer with the second highest incidence among women,[25] and our results show that several laboratories either do not detect or detect a low number of cervical cancer precursor lesions. Therefore, there is a need to invest not only in internal and external quality control of laboratories performing cervical cancer diagnosis, but also in strict inspections of such laboratories by competent authorities, as stated in the Brazilian Ministry of Health Ordinance n. 3,388,[11] issued December 30, 2013, so that statistics regarding cervical cancer may improve in the near future.

Acknowledgments

To the Universidade Federal de Ouro Preto, Minas Gerais Research Foundation (Fundação de Amparo à Pesquisa do Estado de Minas Gerais), the Brazilian Federal Agency for the Support and Evaluation of Graduate Education (Coordenação de Aperfeiçoamento de Pessoal de Nível Superior), and the Brazilian Network for Health Technology Assessment/National Council for Scientific and Technological Development (Rede Brasileira de Avaliação de Tecnologias em Saúde/Conselho Nacional de Desenvolvimento Cientifico e Tecnológico – REBRATS/CNPq).

• References

• 1 Saslow D, Solomon D, Lawson HW , et al; ACS-ASCCP-ASCP Cervical Cancer Guideline Committee. American Cancer Society, American Society for Colposcopy and Cervical Pathology, and American Society for Clinical Pathology screening guidelines for the prevention and early detection of cervical cancer. CA Cancer J Clin 2012; 62 (3) 147-172
• 2 Vale DBAP, Morais SS, Pimenta AL, Zeferino LC. Assessment of the cervical cancer screening in the Family Health Strategy in Amparo, São Paulo State, Brazil. Cad Saude Publica 2010; 26 (2) 383-390
• 3 Gullo CE, Dami AL, Barbosa AP , et al. Results of a control quality strategy in cervical cytology. Einstein (Sao Paulo) 2012; 10 (1) 86-91
• 4 Gajjar K, Ahmadzai AA, Valasoulis G , et al. Histology verification demonstrates that biospectroscopy analysis of cervical cytology identifies underlying disease more accurately than conventional screening: removing the confounder of discordance. PLoS ONE 2014; 9 (1) e82416
• 5 Cuzick J, Clavel C, Petry KU , et al. Overview of the European and North American studies on HPV testing in primary cervical cancer screening. Int J Cancer 2006; 119 (5) 1095-1101
• 6 Michelow P, McKee G, Hlongwane F. Rapid rescreening of cervical smears as a quality control method in a high-risk population. Cytopathology 2006; 17 (3) 110-115
• 7 Azara CZ, Manrique EJ, Alves de Souza NL, Rodrigues AR, Tavares SB, Amaral RG. External quality control of cervical cytopathology: interlaboratory variability. Acta Cytol 2013; 57 (6) 585-590
• 8 Djemli A, Khetani K, Auger M. Rapid prescreening of Papanicolaou smears: a practical and efficient quality control strategy. Cancer 2006; 108 (1) 21-26
• 9 Yunes-Díaz E, Ruiz PA, Lazcano-Ponce E. Assessment of the validity and reproducibility of the Pap smear in Mexico: necessity of a paradigm shift. Arch Med Res 2015; 46 (4) 310-316
• 10 Brasil. Ministério da Saúde. Instituto Nacional de Câncer José Alencar Gomes da Silva. Manual de gestão da qualidade para laboratório de citopatologia. Rio de Janeiro: INCA; 2012
• 11 Brasil. Ministério da Saúde. Portaria n. 3.388, de 30 de dezembro de 2013. Redefine a Qualificação Nacional em Citopatologia na prevenção do câncer do colo do útero (QualiCito), no âmbito da Rede de Atenção à Saúde das Pessoas com Doenças Crônicas. Diário Oficial da União, Brasília (Dec. 31, 2013); Sec. 1:42
• 12 Bortolon PC, Silva MAF, Corrêa FM , et al. Avaliação da qualidade dos laboratórios de citopatologia do colo do útero no Brasil. Rev Bras Cancerol 2012; 58 (3) 435-444
• 13 Brasil. Ministério da Saúde. DATASUS [Internet]. SISCOLO/SISMAMA. 2015 . Available at: http://w3.datasus.gov.br/siscam/index.php . Accessed on Jan 23, 2015
• 14 Pajtler M, Audy-Jurković S, Skopljanac-Macina L, Antulov J, Barisić A, Milicić-Juhas V. Rapid cervicovaginal smear screening: method of quality control and assessing individual cytotechnologist performance. Cytopathology 2006; 17 (3) 121-126
• 15 Confortini M, Di Stefano C, Biggeri A , et al. Daily peer review of abnormal cervical smears in the assessment of individual practice as an additional method of internal quality control. Cytopathology 2014; ; [Epub ahead of print]
• 16 Mattosinho de Castro Ferraz MdaG, Dall' Agnol M, di Loreto C , et al. 100% rapid rescreening for quality assurance in a quality control program in a public health cytologic laboratory. Acta Cytol 2005; 49 (6) 639-643
• 17 International Agency for Research on Cancer. Cervix cancer screening. Lyon: IARC Press; 2005. . (IARC Handbooks of Cancer Prevention, vol. 10)
• 18 Tworek JA, Henry MR, Blond B, Jones BA. College of American Pathologists Gynecologic Cytopathology Quality Consensus Conference on good laboratory practices in gynecologic cytology: background, rationale, and organization. Arch Pathol Lab Med 2013; 137 (2) 158-163
• 19 Davey DD, Neal MH, Wilbur DC, Colgan TJ, Styer PE, Mody DR. Bethesda 2001 implementation and reporting rates: 2003 practices of participants in the College of American Pathologists Interlaboratory Comparison Program in Cervicovaginal Cytology. Arch Pathol Lab Med 2004; 128 (11) 1224-1229
• 20 Fernandes F, Furtado Y, Russomano F , et al. Diagnóstico citopatológico de ASC-US e ASC-H no Serviço Integrado Tecnológico em Citologia do INCA. Rev Bras Cancerol 2012; 58 (3) 453-459
• 21 Eversole GM, Moriarty AT, Schwartz MR , et al. Practices of participants in the college of american pathologists interlaboratory comparison program in cervicovaginal cytology, 2006. Arch Pathol Lab Med 2010; 134 (3) 331-335
• 22 Cormier K, Schaaf M, Hamilton S, Tickman RJ, Perez-Reyes N, Sturgis CD. NILM Pap slides from women 30 years of age and older with positive high-risk HPV DNA. Focused rescreening prior to report issuance, an enhanced quality control measure. Am J Clin Pathol 2014; 141 (4) 494-500
• 23 Ázara CZS, Manrique EJC, Tavares SBN, de Souza NL, Amaral RG. Internal quality control indicators of cervical cytopathology exams performed in laboratories monitored by the External Quality Control Laboratory. Rev Bras Ginecol Obstet 2014; 36 (9) 398-403
• 24 Amaral AF, Araújo ES, Magalhães JC, Silveira EA, Tavares SBN, Amaral RG. [Impact of training about cervical cancer screening on health professionals working in basic health care units]. Rev Bras Ginecol Obstet 2014; 36 (4) 182-187
• 25 Brasil. Ministério da Saúde. Instituto Nacional de Câncer José Alencar Gomes da Silva. Coordenação de Prevenção e Vigilância. Estimativa 2014: incidência de câncer no Brasil. Rio de Janeiro: INCA; 2014