Evaluation of Human Papillomavirus Testing in Primary Screening for Cervical Abnormalities:  Comparison of Sensitivity, Specificity, and Frequency of Referral FREE

Concern about the low sensitivity of the conventional Papanicolaou (Pap) smear has prompted a search for newer methods to either supplement or replace it.^1- 3 A recent review concluded that 47% of women who developed invasive cervical cancer have had adequate screening histories within 5 years of detection.⁴ Some women failed to follow up on an abnormal smear result, while many had a history of negative smear results. The Bethesda system for reading and interpreting cervical cytology specimens was introduced to improve the accuracy of Pap smear diagnoses.³ However, the diagnostic categories introduced by the Bethesda System appear to be no more reproducible than diagnostic categories of other systems.^5- 7 Another concern is the high proportion of smears (over 10% in some populations) read as atypical cells of undetermined significance (ASCUS) and the fact that 5% to 10% of women with this diagnosis have an underlying high-grade lesion.⁸

Human papillomavirus (HPV) is now established as a cause of cervical cancer.^9- 10 In particular, persistent detection of high-risk HPV types is a strong predictor of development of high-grade cervical precancer and invasive cervical cancer.^11- 13

A handful of studies, to date, have examined the role of testing for HPV DNA in primary screening. Most of these studies did not refer women with completely negative test results to colposcopy, so the resulting measures of sensitivity may be overly optimistic. In addition, no measures of specificity can be calculated.^14- 15 One study of 2098 Canadian women, reported by Ratnam et al,¹⁶ did include referral of women with completely negative test results to colposcopy. The estimated sensitivity of HPV DNA testing was higher than that of conventional Pap smear (68.1% vs 40.2%). The estimated specificity of Pap testing was 91.6% compared with 90.6% for HPV DNA testing.

The thin-layer Pap technique was introduced in the early 1990s. It may have a higher sensitivity than the conventional Pap smear and allows for HPV testing of residual liquid that remains after a cytology slide has been made.¹⁷ Despite this improved sensitivity, triage studies of women with Pap smears graded as ASCUS indicate that, compared with a repeat thin-layer Pap smear, a test for high-risk HPV types has a better sensitivity for detecting high-grade lesions or cancer.^8,18 A comparison of the accuracy of thin-layer Pap vs HPV DNA testing in primary screening has not yet been reported.

Testing for HPV DNA may be more specific in older vs younger populations. While HPV DNA is commonly detected in cervical specimens of sexually active women younger than 30 years and may indicate transient infection, it is detected less often in women older than 30 years, where it is more likely to be associated with high-grade lesions or cancer.¹⁴

The present study was undertaken to evaluate the potential of HPV testing for improving screening accuracy for detection of high-grade lesions or cancer among a population of women of reproductive age and to see whether the level of accuracy varies by age. Two different assays for HPV DNA testing were compared: a polymerase chain reaction (PCR)–based assay and a liquid-based DNA-RNA hybridization capture with signal amplification assay (signal amplification).

From December 1997 to October 2000, 4358 consecutive women presenting for annual examinations at 1 of 3 Planned Parenthood clinics in Washington State were invited to participate in the study. Women who were 18 to 50 years of age were eligible if they had no history of hysterectomy, chronic immune suppression, or treatment for cervical neoplasia. Women who provided written informed consent were enrolled. The University of Washington Institutional Review Board approved all protocols.

After consent was obtained, a standardized questionnaire was used to collect demographic, reproductive, and sexual history information. A gynecologic examination was then performed, and screening test samples were obtained according to the following protocol. For cervical cytology, an Ayres spatula was used to collect cells from the transformation zone and a cytobrush was used to collect endocervical cells. Both the brush and the spatula were repeatedly tapped and rinsed in PreservCyt (Cytyc Corp, Boxborough, Mass) for processing the final thin-layer cytology slide (ThinPrep, Cytyc Corp). Next, a specimen for HPV DNA testing was obtained by rotating a dacron-tipped swab in the endocervical os and swabbing it on the ectocervical epithelium. This swab was placed in standard transport medium (STM) (Digene, Gaithersburg, Md). All STM specimens from the screening visit were tested for HPV DNA by PCR. Prior to January 2000, STM specimens from the screening visit of the 712 women who attended colposcopy were tested for HPV DNA by signal amplification (Hybrid Capture 2, Digene). Starting in January of 2000, HPV DNA signal amplification testing of residual thin-layer Pap liquid from all screening visits was introduced (n = 1150). No change was made to the specimen collection protocol.

Before April 1999, a urine sample was collected (n = 1533). Poor sensitivity and a high proportion of inadequate samples (data not presented) led to a decision to stop collecting urine for HPV testing.

Women were referred for colposcopy and biopsy if they had ASCUS, atypical glandular cells of undetermined significance (AGUS), low-grade squamous intraepithelial lesion (LSIL), or high-grade squamous intraepithelial lesion (HSIL) or higher on thin-layer Pap; or a positive PCR test result for high-risk HPV types. A 45% random sample of the first 1000 women with negative Pap and HPV DNA test results was invited to have colposcopy performed. This high percentage was initially selected to determine rates of colposcopy participation. Oversampling for this group continued, however, due to low participation. Of 2631 women with negative cytology and HPV results, 1079 (41%) were randomly selected, and 202 (7.7%) had colposcopy performed. In January 2000, with the initiation of screening by signal amplification, women who were positive for high-risk HPV types by signal amplification were also referred for colposcopy. All women eligible for colposcopy were telephoned a minimum of 3 times and sent a letter when telephone contact failed. The average time between the screening visit and the colposcopy visit was 3 months.

At the colposcopy visit, a detailed standardized questionnaire was completed and cervical samples for thin-layer Pap and HPV DNA testing were obtained as outlined above for the screening visit. One of 2 experienced colposcopists performed colposcopy using the DenVu computerized colposcopy system (DenVu, Tucson, Ariz). Ectocervical biopsies of visible lesions (or the 12 o'clock location if no lesion was visible) were obtained. Endocervical curettage was performed when the lesion extended into the endocervical canal, the Pap test showed HSIL but the lesion was not visible on colposcopy, or the Pap test showed AGUS. Women with histologically confirmed cervical intraepithelial neoplasia (CIN) grade 2 or 3 or invasive cervical cancer were referred for treatment. Women with HSIL on thin-layer Pap and less than CIN 1 on histology underwent repeat biopsy. Women with CIN 1 returned to Planned Parenthood for follow-up care.

Thin-layer slide preparation and processing was performed according to the manufacturer's specifications (Cytyc Corp). The thin-layer slide was then stained with the Papanicolaou stain, screened by a cytotechnologist (certified by Cytyc), and reviewed by pathologists at Harborview Medical Center in Seattle, Wash, having no knowledge of other laboratory or clinical data. Smears were classified as unsatisfactory based on finding more than 60% of the slide target area without epithelial cells. Cellular changes were classified according to the Bethesda classification system as negative, ASCUS, AGUS, LSIL, HSIL, or suggestive of cancer.¹⁹ A random 10% sample of all slides read as normal was rescreened manually as mandated by federal law.²⁰

Hemotoxylin-eosin–stained slides of the biopsy tissue were prepared and reviewed by the pathologist without knowledge of other clinical or laboratory data. Diagnoses were assigned using both the CIN and Bethesda classification systems.

HPV DNA PCR amplification reactions were performed using 5′ biotinylated MY09, MY11, and HMB01 primers (Invitogen Co, Carlsbad, Calif) and Amplitaq Gold polymerase (Applied Biosystems, Foster City, Calif). To prevent PCR product carryover, dTTP was replaced by dUTP and uracil-N-glycosylase was added. The human β-globin gene was coamplified in the HPV reaction mix using 5′ biotinylated primers PC04 and GH20 to monitor specimen adequacy. Two microliters of each specimen was added to 100 µL of reaction mix. Amplification by PCR was carried out in a TC 9600 thermal cycler (Perkin Elmer, Wellesley, Mass) with the following profile: 95°C for 9 minutes to activate the polymerase; 40 cycles at 95°C for 1 minute, 55°C for 1 minute, and 72°C for 1 minute, and a 5-minute terminal extension at 72°C.

HPV DNA typing analysis was performed according to the manufacturer's specifications²¹ using the reverse-line strip test (Roche, Emeryville, Calif) to detect high-risk HPV types 16, 18, 26, 31, 33, 35, 39, 45, 51, 52, 55, 56, 58, 59, 68, 73, 82, or 84.

The signal amplification test was configured to detect in a single assay one or more of the following high-risk HPV types: HPV 16, 18, 31, 33, 35, 39, 45, 51, 52, 56, 58, 59, or 68. Prior to January 2000, the assay was performed according to the manufacturer's specifications on residual screening STM samples (after the aliquot for PCR was removed) from 712 women who returned for colposcopy. Starting in January 2000, testing of residual PreservCyt liquid was performed on all screening samples (n = 1150). A positive result was recorded for specimens with a relative light unit of 1 or greater, corresponding to 5000 or more HPV DNA copies.

As configured and performed in this study, both the PCR and the signal amplification assays had analytic sensitivities of 10 to 100 copies of HPV DNA per sample.

Estimates of sensitivity and specificity for detection of CIN 3 or higher and of the percentage of women referred for colposcopy were obtained for 7 screening strategies. These strategies included referral for colposcopy of women with the following positive test results: (1) a screening thin-layer Pap test showing LSIL or higher or a screening thin-layer Pap test showing ASCUS with a follow-up thin-layer Pap test showing ASCUS or higher; (2) a screening thin-layer Pap test showing ASCUS or higher; (3) a screening thin-layer Pap test showing LSIL or higher or a screening thin-layer Pap test showing ASCUS with a screening PCR test result that was positive for high-risk HPV types; (4) a screening thin-layer Pap test showing LSIL or higher or a screening thin-layer Pap test showing ASCUS with a screening signal amplification test result that was positive for high-risk HPV types; (5) a screening PCR test result that was positive for high-risk HPV types; (6) a screening and follow-up PCR test result that was positive for high-risk HPV types; or (7) a screening signal amplification test result that was positive for high-risk HPV types.

A strategy of repeat testing for high-risk HPV DNA by signal amplification could not be evaluated because only 163 women had these results for both the screening and follow-up visits.

To calculate sensitivity and specificity, 2 × 2 tables were generated using results from the screening and colposcopy visits. Analyses were first performed using the total population and then separately for women younger than 30 years and for women aged 30 years or older. The definition of a positive screening test result was based on the strategy-specific criteria for referral for colposcopy. Depending on the strategy, women with an abnormal Pap result (≥ASCUS) or high-risk HPV DNA were categorized as test positive. Women with negative results (including women who were positive for low-risk HPV types) were categorized as test negative. Since women with inadequate Pap results are usually asked to return for a repeat test, Pap and HPV DNA test results that were inadequate, missing, or insufficient were initially coded as positive. A sensitivity analysis was performed to determine how coding the inadequate, insufficient, and missing results as negative affected the estimates of sensitivity and specificity. Additional sensitivity analyses were based on the use of CIN 2 or higher as the criterion standard, and on separate analyses for signal amplification prior to and after January 2000 to account for differences in specimen collection and testing.

Using the screening results, we defined "screening strata" based on the 6 possible combinations of thin-layer Pap results (ASCUS/LSIL, HSIL, normal) and PCR result (positive/negative for high-risk HPV DNA). For women with normal Pap and negative PCR results, we deliberately invited only a random subsample to return for colposcopy. Analyses indicated that the likelihood of referral for colposcopy depended on the perceived severity of the screening results. Thus, estimates of sensitivity, specificity, and prevalence that ignore these sampling issues will be biased. It is possible to correct for this bias when estimating sensitivities, specificities, and prevalence. Further details in each case are provided below.

To determine the expected number of women testing positive by signal amplification overall and prior to 2000, the number of positive screening signal amplification samples was divided by the sampling fraction for each screening stratum. Summation across all strata provided an estimate of the total number of positive results that would have been obtained if all women screened had been tested by signal amplification.

Uncorrected sensitivity, corrected sensitivity, and corrected specificity were calculated as follows. Uncorrected sensitivity was calculated with the usual formula for sensitivity:

where n_i is the number of true positives who returned for colposcopy in the ith stratum and x_i is the number of those true positives that screen positive. The sums are over 5 of the 6 screening strata. Specifically, we omit the stratum of women who screened negative by both Pap and PCR. Thus, uncorrected sensitivity refers to calculations based only on women who returned for colposcopy and who had a positive screening result for at least 1 of the tests. Such estimates can only be used to determine whether more cases were associated with a positive result on one test than on another test. The lack of outcome information for women with negative results for both tests means that there is incomplete information for the total number of false-negative results and true-negative results for each test (ie, there is verification bias).

Corrected sensitivity was calculated from the following formula:

where f_i is the proportion of those in screening stratum i who return for colposcopy. Corrected specificity was defined analogously.

The corrected sensitivity and specificity estimate the sensitivity and specificity that would have been obtained if all women who were screened had returned for colposcopy. Within each screening stratum, the distribution of lesion grades among those who did and did not return for colposcopy was assumed to be the same. This is a plausible assumption, since HPV DNA and thin-layer Pap results were the only factors found to be independently associated with both return for colposcopy and detection of CIN 3 or higher. Other potentially confounding factors, including age, race/ethnicity, education, income, number of sex partners, hormonal contraceptive use, time between screening and colposcopy visit, and parity, were not independently associated with both colposcopy attendance and detection of CIN 3 or higher (data not shown).

Means, medians, and SDs were calculated for continuous variables. Differences in proportions were tested using the χ² test. Univariate and multivariate logistic regression analyses were performed to estimate odds ratios (ORs) and 95% confidence intervals (CIs) for colposcopy attendance and high-grade histology results. Variances and 95% CIs for corrected sensitivities and specificities were calculated using bootstrap resampling in which the sampling fractions were considered fixed.²² All analyses were performed using STATA 7.0 (STATA Corp, College Station, Tex).

Of 4358 eligible women, 4075 (93.5%) were enrolled. These women were a mean age of 25 years (SD, 5.7), were predominantly white, and reported a lifetime median of 6 sex partners (range, 0-148) (Table 1). Compared with women younger than 30 years (n = 3310), women aged 30 years or older (n = 760) reported a higher lifetime number of sex partners (median of 5 vs 9, respectively), and higher incomes (median of $800 vs $1100 per month, respectively). Older and younger women were similar in terms of race/ethnicity, parity, use of oral contraceptives, and education.

The screening thin-layer Pap results of 678 (16.6%) of 4075 women were abnormal (Table 2). High-risk HPV DNA was detected by PCR in screening samples of 747 (18.3%) women, of whom 231 (31%) women were infected with multiple types (including high- and low-risk types). Among women who were positive for 1 or more high-risk types, HPV type 16 was the most common type detected (28.7%, n = 214), followed by HPV type 52 (13.2%, n = 99) and HPV type 51 (11.4%, n = 85). HPV type 18 was detected in 8.4% (n = 63), HPV type 45 in 4.9% (n = 37), and HPV type 31 in 7.1% (n = 53) of women screening positive for HPV DNA.

Extrapolation of the screening signal amplification results of 712 women who returned for colposcopy to all women screened prior to 2000 provided a high-risk HPV DNA prevalence estimate of 26%. The prevalence estimate for the 1150 women who were screened post-2000, when signal amplification screening results were available for all women, was 28.5%. Since these 2 prevalence estimates were similar, the weighted estimate (27.4%) that was based on a weighted summation of the percentage positive within each screening cytology and PCR stratum was used in calculations of sensitivity and specificity for all strategies that included a signal amplification test.

The percentages of women with different cytologic findings who were positive by PCR for high-risk HPV types were as follows: 82.0% (91/111) with HSIL, 64.4% (104/166) with LSIL, 35.7% (143/401) with ASCUS, 12.0% (398/3318) with normal cytology results, and 11.3% (9/79) with inadequate cytology specimens.The estimated percentages (in the overall population) of women who had positive results by signal amplification were as follows: 85.6% (95/111) for HSIL, 81.9% (136/166) for LSIL, 47.4% (190/401) for ASCUS, and 20.5% (680/3318) for women with normal smear results.

The prevalence of HPV (by PCR) increased in women aged 18 to 24 years and then declined sharply (Figure 1). The prevalence of LSIL declined steadily with increasing age. The prevalence of ASCUS averaged approximately 10% across all age categories. The estimated prevalence of histologically confirmed CIN 3 or higher increased slightly with increasing age.

Of the 1015 women who had biopsies performed, 87 had results of CIN 3 or higher (Figure 2), including 1 woman with endocervical adenocarcinoma in situ. Sixty-four (73.6%) of these lesions were detected in women younger than 30 years. The results for women with lower than CIN 3 were as follows: 50 had CIN 2, 185 had CIN 1, 137 had atypia, and 556 had negative histologic findings.

Three women (ages 30, 32, and 34 years) with CIN 3 or higher diagnosed by biopsy were found to have microinvasive cancer in the specimen from the loop electrosurgical excision procedure and subsequently underwent hysterectomy. All 3 were positive for HPV DNA by signal amplification and had HSIL Pap results at the time of screening. Two of the 3 women were PCR-positive at screening; all 3 were PCR-positive at their colposcopy visit.

The corrected (for colposcopy attendance and verification bias) estimate of CIN 3 or higher prevalence in this population was 3.2%. The 3 screening strategies that were based on testing for high-risk HPV DNA by PCR or by signal amplification were significantly more sensitive than any of the 4 strategies that included thin-layer Pap (84.2%-90.8% vs 57.2%-61.3%) (Table 3). However, the specificity estimates for the HPV DNA detection strategies tended to be lower than for those of the thin-layer Pap–based strategies (72.6%-86.2% vs 82.4%-89.9%). Compared with referral for colposcopy of all women with ASCUS or higher, signal amplification testing of women with ASCUS and referral of those with a positive result was about as sensitive (61.3% vs 60.3%, respectively) and significantly more specific (82.4% vs 88.9%, respectively). A single signal amplification test for high-risk HPV DNA referred the highest proportion of women for colposcopy. Compared with strategies that required repeat Pap or HPV DNA testing of women with ASCUS, the referral strategy based on repeated PCR positivity for high-risk types was significantly more sensitive (84.2% vs 57.2%-60.3%) but significantly less specific (86.2% vs 88.9%-89.9%).

Although the differences were not significant, all screening strategies were more sensitive for CIN 3 or higher among women younger than 30 years than for women aged 30 years or older (Table 4). Estimates of specificity, however, were significantly higher for women aged 30 years or older than for those who were younger than 30 years. Similar results were obtained when CIN 2 or higher was used as the definition of high-grade disease, and again all sensitivity estimates for those aged 30 years or older were lower than for those younger than 30 years.

Estimates of sensitivity and specificity were not appreciably affected by coding inadequate or missing screening test results as negative instead of positive. Lastly, estimates of sensitivity and specificity for strategies that included signal amplification prior to and after 2000 were not significantly different (data not shown).

In this first US-based study comparing screening by thin-layer Pap smear to screening by HPV DNA tests for detection of CIN 3 or higher, HPV DNA testing was significantly more sensitive, but significantly less specific, than thin-layer Pap testing. Compared with referring all women with abnormal smear results for colposcopy, strategies that incorporated HPV DNA testing of ASCUS resulted in significant improvements in specificity, with only slight decreases in sensitivity. The strategy of requiring 2 positive PCR tests for HPV DNA was both more sensitive and specific than referral of women with abnormal smear results for colposcopy. All results were robust to differences in assumptions regarding the definition of high-grade disease, age of the screening population, and coding of missing or unsatisfactory screening test results.

Our uncorrected sensitivity estimates (for detection of CIN 2 or higher) were generally similar to estimates presented in other studies.¹⁵ A comparison of our results with those from a study of 2098 Canadian women (which included referral of women with negative screening results)¹⁶ showed similar corrected sensitivities for Pap tests (47.5% vs 40.2%) and for signal amplification HPV DNA tests (71.4% vs 68.1%), but lower corrected specificity estimates for Pap testing (82.9% vs 91.6%), and for signal amplification testing (73.4% vs 90.6%). These differences may be due in part to the fact that, in the Canadian study, biopsies were obtained at the discretion of the colposcopist, women older than 50 years were included, conventional Pap smears were used, and a less-sensitive signal amplification test was used for most of the study.

Overall, more women were positive for high-risk HPV DNA by signal amplification (27.4%) than by PCR (18.3%). Although a similar proportion of women with HSIL were positive by signal amplification and by PCR, more women with LSIL, ASCUS, or normal cytology results were positive by signal amplification than by PCR. One explanation for this difference is that the signal amplification test for high-risk HPV types will detect the low- and indeterminate-risk types that are frequently found in samples from women with normal, ASCUS, or LSIL Pap results, but not in samples from women with HSIL.^23- 25 Limiting HPV testing to women aged 30 to 50 years resulted in improved specificity. However, thin-layer Pap testing was also more specific, and both HPV testing and thin-layer Pap testing were less sensitive in the older population. Due to the age-related shift of the squamocolumnar junction from the ectocervix to the endocervix, sample collection for both Pap and HPV DNA tests may be more difficult among older vs younger women.

The characteristics of our population were similar to those of the population enrolled in the ASCUS/LSIL Triage Study (ALTS),⁸ a multicenter study of triage strategies for women with ASCUS or LSIL sponsored by the National Institutes of Health. In particular, the CIN 3 or higher prevalence among 401 women with ASCUS in our study was comparable to the CIN 3 or higher prevalence reported for 1149 women with ASCUS enrolled in ALTS (5.2% vs 5.1%, respectively), where histologic outcomes were determined by an expert pathology panel.

Despite differences in age and ethnicity between our Planned Parenthood population and the population of women from health maintenance organizations studied by Manos et al,¹⁸ both studies showed that testing of women with ASCUS for high-risk HPV DNA by signal amplification was more sensitive for detection of CIN 2 or higher than was a repeat Pap test showing ASCUS or higher. Our study extends these findings by showing that, compared with referring all women with ASCUS to colposcopy, referring only those with positive signal amplification test results achieved a significant improvement in specificity, with only a slight loss in sensitivity. Whether estimates of sensitivity and specificity for all of the 7 screening strategies can be reproduced in older and less sexually active populations remains to be determined.

Our most serious potential limitation was low colposcopy attendance. However, prior studies of screening in the United States have observed nonattendance of 23% to more than 60%.^26- 27 In our study, women who returned for colposcopy tended to have more severe Pap smear findings and positive HPV test results. To address the potential bias arising from differences in colposcopy attendance for women with different test results, our estimates of sensitivity and specificity were corrected. Importantly, while there were changes in the value of estimates after correction, the overall conclusions did not change. Additionally, none of the women with completely negative screening results who returned for colposcopy had CIN 3 or higher, suggesting that referral based on high-risk HPV DNA testing in conjunction with thin-layer Pap testing identified most, if not all, women with CIN 3 or higher, although our sample of women with completely negative results was relatively small.

In our study, a higher percentage of women with abnormal Pap results (72.7%) than women with positive HPV DNA results (67.9%) complied with recommended follow-up. Schneider et al²⁸ reported reversed percentages, with 80.1% of women with positive HPV DNA tests and 61.9% of women with abnormal cytology findings complying with recommended follow-up. When a significant portion of a referred population fails to comply with follow-up, it suggests that patients and/or their clinicians are uncertain as to the significance of an abnormal or positive screening test result.²⁹ Better tests, clearer guidelines, and more education will help.

Cervical cancer screening practices are quite variable. In settings where programs invite women to participate and virtually all women are screened at regular intervals, the costs (psychological and monetary) of screening with a less-specific test may outweigh the risk of delayed treatment associated with use of a less-sensitive test. In settings where screening is more haphazard than routine, or performed infrequently, screening with a highly sensitive test provides greater reassurance that disease has not been missed in women who screen negative.

In summary, a single HPV DNA test was more sensitive but less specific for detection of CIN 3 or higher than any of the cytology-based strategies. All screening strategies were somewhat less sensitive and significantly more specific among women aged 30 years and older vs women younger than 30 years. In some settings, screening for high-risk HPV DNA may be a reasonable alternative to cytology-based screening.