ABSTRACT
The objective was to assess the diagnostic test accuracy of high-risk human papillomavirus (hrHPV) testing of self-collected urine and cervicovaginal samples for the detection of cervical intraepithelial neoplasia grade 2 or higher (CIN2+). We recruited a convenience sample of women 25 to 65 years of age who were undergoing clinically indicated colposcopy at two medical centers in North Carolina between November 2016 and January 2019. Women with normal cytology results and positive hrHPV results were also recruited. Urine samples, self-collected cervicovaginal samples, provider-collected cervical samples, and cervical biopsy samples were obtained from all enrolled women. Samples were tested for hrHPV DNA using the Onclarity assay (Becton Dickinson, Sparks, MD). Biopsy samples were histologically graded as CIN2+ or <CIN2. We calculated the sensitivity and specificity for detection of CIN2+ and assessed agreement between sample collection methods. We included 307 women (median age, 36 years) with valid histology results and triple-matched urine, self-collected cervicovaginal, and provider-collected cervical hrHPV results; 83 women (27%) had CIN2+. Urine-based hrHPV testing correctly identified 80% of CIN2+ cases (95% confidence interval [CI], 71 to 88%) using the PCR cycle threshold (CT) established for provider-collected cervical samples, but sensitivity remained below the estimates for self-collected cervicovaginal and provider-collected cervical samples (both 94% [95% CI, 89 to 99%]). Using a higher CT cutoff value of ≤40, 90% sensitivity was achieved for urine-based hrHPV testing. Agreement between results for urine samples and self-collected cervicovaginal samples (kappa = 0.58) or provider-collected cervical samples (kappa = 0.54) was moderate. Urine-based hrHPV testing may be a promising approach to improve cervical cancer screening coverage, especially among women with limited access to health care.
INTRODUCTION
Although invasive cervical cancer (ICC) is preventable through human papillomavirus (HPV) vaccination, as well as screening and treatment of precancerous cervical lesions, the global ICC burden remains high (1). U.S. guidelines currently recommend screening with cytology alone, primary high-risk HPV (hrHPV) testing, or cytology-hrHPV cotesting every 3 to 5 years, depending on the woman’s age (2–5). Unfortunately, not all women are screened at the recommended intervals, and most ICC cases occur among underscreened women (6–8). Barriers to screening include cost, inflexible working hours, lack of transportation and childcare, fear of pain, and embarrassment (9, 10). New screening approaches addressing these barriers are needed to improve cervical cancer prevention.
While cytology requires a provider-collected cervical sample, hrHPV testing can be performed on self-collected specimens. Noninvasive, urine-based hrHPV testing might be especially attractive for women who are reluctant to undergo gynecological examinations or to perform cervicovaginal self-sampling. For the detection of cervical intraepithelial neoplasia grade 2 or higher (CIN2+), self-collected cervicovaginal samples and provider-collected samples for hrHPV testing are similarly sensitive (11, 12). The performance of urine-based hrHPV testing for CIN2+ detection is less clear, because studies have used various hrHPV assays and nonstandardized urine collection methods (13–22).
Our objective was to assess the accuracy of urine-based hrHPV testing for CIN2+ detection using media to preserve urine samples. We compared the accuracy of urine-based hrHPV testing to testing of self-collected cervicovaginal and provider-collected cervical samples and assessed the agreement between different sampling methods.
(Preliminary results of this work were presented at EUROGIN 2018, Lisbon, Portugal, 2 to 5 December 2018.)
MATERIALS AND METHODS
Study population.Between November 2016 and January 2019, we recruited a convenience sample of women, 25 to 65 years of age, who were attending colposcopy clinics at the University of North Carolina at Chapel Hill (UNC) Women’s Hospital or Duke University Hospital for one of the following indications: abnormal cytology results, infection with HPV-16/18, persistent infection with other hrHPV subtypes, or treatment for CIN2+. We also aimed to include a sample of HPV-positive women at lower risk for CIN2+, who would be more similar to a primary screening population than the patients undergoing clinically indicated colposcopy. Therefore, we invited women with normal cytology findings and positive hrHPV results for HPV types other than HPV-16/18 at the time of their routine screening. This group was referred to as “research only,” because current U.S. guidelines do not recommend immediate referral for colposcopy for such women (4).
Potentially eligible women were identified through review of electronic medical records and were contacted via telephone or during their clinic visits. Women were excluded if they were pregnant or had had their cervix removed; additionally, women in the research only group were excluded if they were taking anticoagulants or if the enrollment date was not within 3 months after their original hrHPV diagnosis. Written informed consent was obtained from all participating women. The study was approved by the institutional review boards at UNC and Duke University.
Sample collection.During the clinic visit, participating women received detailed verbal and written instructions, in English or Spanish, concerning the study procedures. The women then provided two urine samples, i.e., an initial-stream sample of ∼20 ml for hrHPV testing and a midstream sample of up to 100 ml for pregnancy testing, if clinically indicated. Next, the women self-collected a cervicovaginal sample by inserting a Viba brush (Rovers Medical Devices BV, Oss, The Netherlands) to the top of the vaginal canal, rotating it five times, and releasing the brush head into a vial prefilled with 6 ml of preservative liquid-based cytology medium (ThinPrep; Hologic Inc., Bedford, MA).
Next, a pelvic examination was performed by a clinical provider, during which a cervical scraping was collected with two 360° turns, in a clockwise fashion, of a brush-like cervical cell collector (Wallach Papette; Wallach Surgical Devices, Trumbull, CT). The provider-collected cervical sample was preserved in a 20-ml vial of ThinPrep medium for subsequent hrHPV testing. Colposcopy was performed for all women, following cervical treatment with 3 to 5% acetic acid (followed by Lugol’s iodine at the Duke University Hospital site), according to standard clinical procedures. Directed biopsy samples were taken from visible cervical lesions, and endocervical curettage (ECC) was performed if the transformation zone or the limits of a lesion near the cervical os could not be fully visualized. If no cervical lesions were observable, then one random biopsy sample was taken at the 12 o’clock position of the cervix and ECC was performed. At least two cervical biopsy samples were obtained during each procedure. The loop electrosurgical excision procedure was performed when clinically indicated. At the end of the visit, the women received a gift card ($40 for women who had undergone clinically indicated colposcopy and $120 for research only participants).
Sample processing and laboratory analyses.From the initial-stream specimen, 2 ml of urine was transferred (within 5 min after collection) into a Becton Dickinson (BD) molecular tube containing 0.2 ml of a proprietary preservative to ensure sample stability. The tubes are fitted with a pierceable cap to facilitate automated sample processing on the BD Viper LT System. All samples and the BD molecular tube were placed in a cooler with frozen gel packs within 10 min after sample collection and were kept cool until they could be further aliquoted the same day. The self- and provider-collected samples were vortex-mixed for 10 to 30 s, and 0.5 ml of each sample was transferred to separate BD molecular tubes containing 1.7 ml of an HPV diluent buffer. All three BD molecular tubes were stored at –20°C until they were shipped to BD for hrHPV testing using the Onclarity assay (BD, Sparks, MD). The staff at BD did not have access to any clinical information concerning the participants. The Onclarity assay uses PCR and nucleic acid hybridization to detect DNA of 14 hrHPV subtypes. Six hrHPV subtypes (subtypes 16, 18, 31, 45, 51, and 52) are individually genotyped, and the other 8 subtypes are identified in three groups (subtypes 33/58, 56/59/66, and 35/39/68). Provider- and self-collected Hologic ThinPrep specimens were processed using the standard liquid-based cytology workflow on the BD Viper LT System. The Onclarity assay has been approved by the FDA for use with provider-collected samples in BD SurePath preservative fluid, but it shows similar performance when used with ThinPrep PreservCyt transport medium (23). Urine samples were processed using a co-collection device workflow, without a prewarming step. The remaining provider-collected cervical sample was sent to the UNC cytopathology laboratory for cytological analysis, if clinically indicated, or storage. The Onclarity hrHPV test results were obtained for research purposes only and were not shared with participants.
Histological diagnoses (<CIN2 versus CIN2+) served as the reference standard for this test accuracy study. Cervical biopsy samples from women who underwent colposcopy as part of their scheduled clinical appointments were sent to the UNC or Duke University Hospital surgical pathology laboratory for histological evaluation, according to standard procedures. Pathologists had access to clinical information captured in the electronic medical records but were unaware of the study hrHPV results. The colposcopy patients were informed of the histological results by the clinical team responsible for their care. Biopsy samples taken from women in the research only group were analyzed by a gynecological pathologist at the UNC translational pathology laboratory, who did not have access to clinical information. Women in the research only group were contacted with histological results by the study team after review by a practicing gynecologist. Women whose samples were inadequate for pathology review were invited to return for additional sampling. Women with CIN2+ were referred for further treatment, according to the standard of care.
Sample size considerations, definitions, and statistical analyses.Study recruitment was guided by an expected CIN2+ prevalence of 31% among the colposcopy clinic patients (13) and 6% among the research only participants (24). Our sample size calculations were focused on estimating the sensitivity of urine-based hrHPV testing with adequate precision. We expected 90% sensitivity and aimed to achieve sampling error margins of 10 percentage points. Thus, we aimed to recruit 160 colposcopy patients (49 CIN2+ cases expected) and 250 research only participants (15 CIN2+ cases expected). However, recruitment of research only participants turned out to be logistically challenging, with 79% of eligible women declining participation, cancelling clinic visits, or not being reachable. Therefore, we recruited more colposcopy patients and fewer research only participants than planned.
We included women with valid triple-matched results (urine, self-collected cervicovaginal, and provider-collected cervical samples available) in the test accuracy analysis; participants with missing samples or invalid test results were excluded. Participants without a valid cervical biopsy sample result (e.g., due to insufficient tissue) were also excluded. There is no established PCR cycle threshold (CT) for the Onclarity assay in urine; therefore, we used the same cutoff value as for provider-collected cervical samples (≤34.2 for all hrHPV subtypes except HPV-16, with a CT of ≤38.3). In an exploratory analysis, we evaluated the effects of different CT cutoff values (≤36.0, ≤38.3, and ≤40.0) on the accuracy of urine-based hrHPV testing for CIN2+. Samples that were reactive for any hrHPV subtypes detected by the Onclarity assay were regarded as positive.
We used descriptive statistics to assess the sociodemographic characteristics of included women. We calculated the sensitivity and specificity of hrHPV testing for CIN2+ detection for different sample types. McNemar’s test was used to assess whether the sensitivity of hrHPV testing with urine samples was different from that with self-collected cervicovaginal samples and provider-collected cervical samples. The level of statistical significance was set at <0.05. We computed the overall agreement and the specific positive agreement and negative agreement between hrHPV test results for urine samples and those for self-collected cervicovaginal samples or provider-collected cervical samples (25). We calculated the unweighted Cohen’s kappa and its 95% confidence interval (CI) to assess the degree of agreement beyond chance between the different sample types. The following interpretation was used: ≤0, no agreement; 0.01 to 0.20, slight agreement; 0.21 to 0.40, fair agreement; 0.41 to 0.60, moderate agreement; 0.61 to 0.80, substantial agreement; 0.81 to 1.00, almost perfect agreement (26).
In sensitivity analyses, we examined whether the test accuracy and agreement results changed if we restricted the study population to patients with clinically indicated colposcopy or lower-risk research only participants. We also computed test accuracy with all valid results for a given sample type (403 results for urine samples, 380 results for cervicovaginal samples, and 343 results for provider-collected cervical samples), and we estimated hrHPV test accuracy for CIN3+ detection. Analyses were performed using SAS/STAT software (SAS Institute Inc., Cary, NC).
Data availability.Study data will be shared upon request, in accordance with the National Institutes of Health (NIH) guidelines on data sharing, as proposed in our original grant submission.
RESULTS
Study population.A total of 434 women (363 with clinically indicated colposcopy and 71 in the research only group) were enrolled (Fig. 1). Of those women, 413 women had valid histology results. In the test accuracy analysis, we included 307 women with valid histology results and valid results for their triple-matched urine, self-collected cervicovaginal, and provider-collected cervical samples. The median age of these women was 36 years (interquartile range [IQR], 31 to 45 years). The study population was ethnically and racially diverse, with 38% non-Hispanic white women (n = 117), 29% Hispanic women (n = 90), and 26% non-Hispanic black women (n = 79) (Table 1). Participant health insurance coverage ranged from private (n = 118 [39%]) or government-sponsored (n = 70 [23%]) to no health insurance (n = 116 [38%]).
Flow diagram showing the number of women eligible for inclusion during the study period and the number of women excluded from the analysis, with reasons.
Baseline characteristics of women included in the test accuracy analysis, with valid urine samples, self-collected cervicovaginal samples, and provider-collected cervical samples
Diagnostic test accuracy for CIN2+ detection.Of 307 included women, 83 (27%) had histologically confirmed CIN2+ (34 CIN2 cases, 11 CIN2/3 cases, 36 CIN3 cases, and 2 ICC cases) and 224 (73%) had <CIN2 (57 CIN1 cases and 167 cases without dysplasia). The most common hrHPV subtypes detected in urine samples were HPV-16, with 58 positive samples (19%), HPV-56/59/66, with 56 samples positive for at least one of these subtypes (18%), and HPV-35/39/68, with 35 samples positive for at least one of these subtypes (11%). HPV-18 was detected in 17 urine samples (6%), and HPV-45 was detected in 18 urine samples (6%). Urine-based hrHPV testing correctly identified 66 of 83 CIN2+ cases when the CT cutoff value for provider-collected samples was used. The sensitivity of hrHPV testing in urine samples (80.0% [95% CI, 70.8 to 88.2%]) was significantly lower than that in self-collected cervicovaginal samples or provider-collected cervical samples (both 94.0% [95% CI, 88.9 to 99.1%]; P = 0.001). The specificity of hrHPV testing was the same in urine samples and provider-collected cervical samples (38.4% [95% CI, 32.0 to 44.8%]), but it was lower in self-collected cervicovaginal samples (30.0% [95% CI, 23.9 to 35.9%]; P = 0.004) (Table 2).
Diagnostic test accuracy of the Onclarity hrHPV assay for detection of CIN2+ with different sample types, using the Onclarity CT cutoff value established for provider-collected samples for all sample types
Table 3 shows test accuracy results for urine-based hrHPV testing using different CT cutoff values. When we chose a higher CT cutoff value of ≤40.0, we correctly identified an additional 9 CIN2+ cases and the sensitivity estimate increased to 90.4% (95% CI, 84.0 to 96.7%), which was similar to the sensitivity estimates for self-collected cervicovaginal samples (P = 0.37) and provider-collected cervical samples (P = 0.45). However, at this cutoff value the assay also yielded positive hrHPV results for an additional 37 <CIN2 cases and the specificity thus decreased to 21.9% (95% CI, 16.5 to 27.3%).
Test accuracy of the urine-based Onclarity hrHPV assay for detection of CIN2+ using different assay cutoff values
Agreement between hrHPV results in different samples.Agreement results are based on the CT cutoff value for provider-collected samples. Overall agreement between hrHPV results in urine and self-collected cervicovaginal samples was 83% (95% CI, 79 to 87%), with positive agreement of 88% (95% CI, 85 to 91%) and negative agreement of 70% (95% CI, 62 to 77%); agreement beyond chance was moderate (kappa = 0.58 [95% CI, 0.48 to 0.68]). Results were similar for the agreement between hrHPV results in urine and provider-collected cervical samples (Table 4).
Agreement between Onclarity hrHPV test results for urine samples and those for self-collected cervicovaginal samples or provider-collected cervical samples
Sensitivity analyses.When we restricted the analyses to 243 patients with clinically indicated colposcopy and a CIN2+ prevalence of 33% (81 CIN2+ cases), the results were similar (see Table S1 in the supplemental material). Agreement between hrHPV test results in urine samples and self-collected cervicovaginal samples or provider-collected cervical samples remained moderate (Table S2). Results limited to the 64 research only participants were imprecise, due to the small sample size and the small number of CIN2+ cases (n = 2 [3%]) (Tables S3 and S4).
When we included all valid results for a given sample type in the test accuracy analysis (403 results for urine samples, 380 results for self-collected cervicovaginal samples, and 343 results for provider-collected cervical samples) (Table S5), the results were similar to those of the main analysis. When we defined CIN3+ as the target condition, the sensitivity estimates increased marginally (Table S6). Urine-based hrHPV testing was 83.7% sensitive (95% CI, 73.3 to 94.0%) for CIN3+ detection.
DISCUSSION
We found that urine-based Onclarity hrHPV testing correctly identified 80% of CIN2+ cases using the CT cutoff value for provider-collected cervical samples, but sensitivity remained below the estimates for self-collected cervicovaginal samples and provider-collected samples (both 94%). When we changed the CT cutoff value for urine samples to ≤40, 90% sensitivity was achieved. Using the CT cutoff value for provider-collected samples, agreement between hrHPV results for urine samples and those for self-collected cervicovaginal samples or provider-collected cervical samples was moderate.
To our knowledge, this is the first study assessing the accuracy of urine-based hrHPV testing for CIN2+ detection using the BD Onclarity assay. Several studies have examined the accuracy of urine-based hrHPV testing for CIN2+ detection using other assays (13, 14, 16–22, 27–29), and most studies have focused on the analytical accuracy of urine-based testing for detection of hrHPV infection (30). Test accuracy estimates vary substantially between studies. An early study evaluated the Hybrid Capture II assay and found low sensitivity of 45% and specificity of 70% for urine hrHPV testing for CIN2+ detection (16). Our findings are in line with later studies that showed higher sensitivities of 80 to 100% and low specificities of 25 to 53% among patients referred for colposcopy, a population at higher risk of HPV infection and CIN2+ (13, 14, 18–21, 28, 29). The more recent studies, including ours, added preservation solutions to stabilize urine samples. One study used the Aptima HPV RNA test (Hologic, Inc.) and found a lower sensitivity of 45% and a specificity of 62% for CIN2+ (17). To date, most studies have been conducted among women referred for colposcopy, and evidence is very limited regarding the performance of urine-based hrHPV testing for CIN2+ detection in primary screening populations (27). The specificity of hrHPV testing is expected to be lower among colposcopy patients than in screening populations, as specificity decreases with increasing hrHPV prevalence (31).
In our study, the sensitivity of urine-based hrHPV testing for CIN2+ detection was relatively lower than that for self-collected cervicovaginal samples and provider-collected samples when the CT cutoff value for provider-collected cervical samples was used. Currently, there is no established CT cutoff value for urine-based Onclarity hrHPV testing. At a CT cutoff value of ≤40, the sensitivity of urine-based hrHPV testing increased to 90% for detection of CIN2+, but the specificity decreased to 22%. Additional investigations are needed to determine the optimal CT cutoff value for the Onclarity assay for urine samples and to further increase the test accuracy of urine-based hrHPV testing. Of note, we attempted to optimize the urine collection process by taking initial-stream urine and quickly adding a preservative solution to stabilize the samples (32). However, the cup-based collection of initial-stream urine, followed by the addition of preservative, may not be as efficient as collecting initial-stream urine with a cup/device that immediately mixes it with preservative.
Strengths of our study include the relatively large sample size and standardized sample collection procedures. We instructed women to collect ∼20 ml of initial-stream urine and then transferred a fixed amount of 2 ml into a BD molecular tube containing 0.2 ml of a proprietary preservative, to obtain similar dilutions for all samples. To avoid verification bias, we aimed to perform the reference test (histology) for all participants and included only women with valid histology results in the test accuracy analysis. We recruited both high-risk women referred for colposcopy and women with normal cytology results and hrHPV infection with other than subtypes 16/18 (the research only group). In a sensitivity analysis, we assessed whether results changed if we restricted our population to women referred for colposcopy; this was not the case. Unfortunately, recruitment of research only participants was logistically challenging, and we could not reach a large enough sample size to draw meaningful conclusions regarding urine validation for this group separately. Another limitation of our study is that the results we found in a CIN2+-enriched population are not necessarily generalizable to a primary screening population.
Although further work is needed to improve its accuracy, urine hrHPV testing has the potential to increase screening options for underscreened women (33). Cost-effectiveness analyses from a program perspective are also warranted, to determine whether urine-based hrHPV screening may be preferred over hrHPV testing based on provider-collected cervical samples or self-collected cervicovaginal samples. Noninvasive urine hrHPV samples seem to be preferred by women over both provider-collected cervical samples (13, 16) and self-collected cervicovaginal samples (13, 16, 22). Furthermore, urine samples could be collected at home and sent by mail to the laboratory for hrHPV testing, thereby removing the need for an initial clinic-based pelvic examination. In a recent study, hrHPV prevalence rates were similar for urine samples collected at home or in the clinic, and most participants were comfortable receiving a urine collection kit in the mail (13).
Urine-based hrHPV tests have not yet been approved by the FDA for clinical use, and further studies to evaluate and to improve the accuracy of urine-based hrHPV testing for CIN2+ detection in a primary screening population and among women living with HIV are needed. Nevertheless, urine-based hrHPV testing may be a viable option to improve cervical cancer screening coverage in the future, especially among women with limited access to health care.
ACKNOWLEDGMENTS
We are grateful to all included women for their participation in the study. We thank Anna Baker, Johana Bravo, Elena DiRosa, Sara Smith, and Claire Edelman for the day-to-day coordination of this study, including the recruitment of participants and data collection, and the staff members of the UNC and Duke University colposcopy clinics who supported this study. We are grateful to Siobhan O’Connor for reviewing the histopathology slides for the research only participants. Furthermore, we thank the UNC Center for AIDS Research HIV/STD Laboratory Core for sample processing and storage and clinical trial support. We also thank BD for donating the Onclarity HPV testing kits, Hologic, Inc., for donating preservation media, cervical sample collection devices, and imager slides, and Rovers Medical Devices for donating self-collection brushes.
This research was supported by NIH grant U54 CA156733 to J.S.S. and the UNC Center for AIDS Research, an NIH-funded program (grant P30 AI50410). E.R. was supported by a grant from the Swiss Cancer Research Foundation (grant BIL KFS-4423-02-2018). Additional support to A.K.K. was provided by the Division of General Obstetrics and Gynecology Women’s Health Research Fellowship. Specimen transport tubes and HPV testing were donated by BD, self-collection brushes were donated by Rovers Medical Devices BV, and cytology medium was donated by Hologic. The content of this publication is solely the responsibility of the authors and does not necessarily represent the official views of the funders.
J.S.S. has received research grants, supply donations, and consultancies from, served on paid advisory boards for, and/or been a paid speaker for Arbor Vita, BD, Hologic, Rovers Medical Devices, and Trovagene in the past 5 years. J.A.E.N. has received financial support from Hologic for work on a different study. L.V., K.C., and B.F. are employed by BD, which sells and distributes cervical cancer screening products, including the BD Onclarity HPV assay.
FOOTNOTES
- Received 1 September 2019.
- Returned for modification 30 September 2019.
- Accepted 12 December 2019.
- Accepted manuscript posted online 2 January 2020.
Supplemental material is available online only.
- Copyright © 2020 American Society for Microbiology.