ABSTRACT
Inexpensive and easy-to-perform human papillomavirus (HPV) tests are needed for primary cervical cancer screening in lower-resource regions. In a convenience sample of 516 residual exfoliative cervical specimens from the Kaiser Permanente Northern California and U.S. National Cancer Institute Persistence and Progression Study, we assessed the agreement and clinical performance of a simple, inexpensive real-time PCR assay for the detection of 13 carcinogenic HPV types (the H13 assay; Hybribio, Hong Kong) that is marketed in limited-resource settings compared to previous testing by the Hybrid Capture 2 assay (HC2; Qiagen, Germantown, MD) and the Onclarity assay (BD Diagnostics, Sparks, MD). The test set was chosen to include many HPV-positive specimens. The reference standard was a combination of HC2 and Onclarity results for HPV detection and histologic diagnosis of controls (less than cervical intraepithelial neoplasia grade 2 [<CIN2]) or cases (cervical intraepithelial neoplasia grade 2 or higher [CIN2+]) for disease status. In this enriched convenience sample, H13 tested positive for 94.4% of the 108 HC2- and Onclarity-positive CIN2+ specimens and negative for 88.2% of the 51 HC2- and Onclarity-negative <CIN2 specimens. H13 positivity was significantly lower than that of HC2 among women with CIN2+ (89.9% versus 98.7%, respectively) (P < 0.001) and <CIN2 (53.5% versus 72.4%, respectively) (P < 0.001). In conclusion, H13 corresponds well to the combination of HC2 and Onclarity and has good clinical accuracy compared to histologic diagnosis, with less cross-reactivity with untargeted HPV types than HC2. H13 is a lower-cost HPV DNA test that might be useful for primary screening in limited-resource settings.
INTRODUCTION
With the recognition that persistent infection with human papillomavirus (HPV) is the main factor leading to the development of cervical cancer, technological advances in the last 2 decades have made vaccination the major primary prevention strategy. Moreover, testing for high-risk HPV DNA is a complementary strategy for secondary cervical cancer prevention that can be integrated with vaccination to control cervical cancer much faster than vaccination alone (1).
HPV testing is more reliable and less dependent on human expertise than visual inspection with acetic acid (VIA) or cytology (Pap tests) (2). Even more important, HPV testing is more sensitive than VIA or cytology for the detection of cervical cancer/precancer, and a negative result provides longer reassurance to the woman that she in not at substantial risk of developing cervical cancer (3, 4). As a third advantage, the possibility of self-collecting samples for HPV testing might increase the acceptability of cervical screening in areas where genital contact is a taboo issue while allowing screening for women that cannot access health facilities (5, 6).
The use of relatively inexpensive and partially automated HPV tests such as real-time PCR (qPCR), which provides results within a few hours, might be practical in low-resource settings. Cost and ease of use ultimately determine the feasibility of incorporating a screening test into cervical cancer prevention strategies in low-resource areas (2), but accuracy comparable to that of well-validated tests must be maintained.
The H13 HPV test (H13; Hybribio Biotechnology Limited Corp., Hong Kong, China) is a recently developed low-cost qPCR-based assay that identifies indistinctly the 13 IARC-recognized oncogenic HPV types. Given its affordability and relative ease of use, the H13 assay is currently used in various cervical cancer screening programs in Asia, West Africa, and Latin America. The test uses viral type-specific primers in the E6 gene. In the standard H13 test procedure, a cell lysate is subjected to real-time PCR amplification using HPV E6 gene PCR primers labeled with the 6-carboxyfluorescein (FAM) fluorescent dye and an internal control amplifying a human DNA sequence, labeled with the HEX (6-carboxy-2,4,4,5,7,7-hexachlorofluorescein)/JOE (6-carboxy-4′,5′-dichloro-2′,7′-dimethoxyfluorescein) dye. The results are manually interpreted using the guide provided by the manufacturer. The method does not require extensive training for the machine's user, and results can be provided within 2 to 3 h (Hybribio, Hong Kong). Thus, it merits consideration for low-resource regions as an alternative to other simpler assays such as careHPV (Qiagen).
For validation, two reference HPV assays for cervical cancer screening were included, Hybrid Capture 2 (HC2) (Qiagen, Germantown, MD) and Onclarity (BD; BD Diagnostics, Sparks, MD). These two established HPV tests are roughly comparable in terms of analytic and clinical sensitivity (7) but are relatively costly and require some laboratory skills to be performed.
In a comparison of the accuracies of H13 and HC2 for the detection of high-grade cervical lesions, a preliminary study in China found that the agreement of the two assays in identifying high-risk HPV types was good (kappa = 0.79) (8). However, as far as we know, no previous study has formally examined the validity of H13 for cervical cancer screening in comparison with two certified assays (HC2 and Onclarity) or assessed the ability of H13 to cross-react with untargeted types. Since H13 is marketed in limited-resource settings, establishing its concordance with existing approved tests and its accuracy in detecting histologically confirmed cervical precancer (the target of cervical screening) will help determine whether its use is appropriate in these settings.
RESULTS
A total of 516 residual specimens selected from women previously tested with HC2 and Onclarity, and aged 20 to 67 years at enrollment into the Kaiser Permanente Northern California and U.S. National Cancer Institute (NCI) Persistence and Progression Study (KPNC PaP) cohort, were retested for high-risk HPV with the new assay, H13. Of these residual specimens, 143 (27.7%) were negative for the internal control and therefore failed to give a valid result with H13. Compared to specimens with valid H13 results, specimens with failed H13 results were more often classified as HC2 negative (HC2−)/Onclarity negative (BD−) (13.9% versus 28.7%, P < 0.001), in line with specimen inadequacy or DNA degradation (Table 1). With this exception, other variables were comparable between samples with valid H13 results and samples with failed H13 results, suggesting that failure to detect HPV with the internal control probe of the H13 assay happened independently of age, cytology, or histology status.
Description of the study population in relation to H13 assay validity
Among participants with valid results, H13 results were positive in 107 (89.9%) and 136 (53.5%) of 119 women with cervical intraepithelial neoplasia grade 2 or higher (CIN2+; cases) and 254 women with less than cervical intraepithelial neoplasia grade 2 (<CIN2; controls), respectively. Among cases, H13 found positive 94.4% of 108 samples that tested positive by HC2 and Onclarity (HC2+/BD+) and 50% of 10 samples with discrepant HC2 and Onclarity results (HC2+/BD− or HC2−/BD+) (Table 2). The only case sample that tested negative by HC2 and Onclarity (HC2−/BD−) was also found negative by H13. Among controls, 91.2% of 102 HC2+/BD+ samples were found positive by H13, as were 36.6% of 101 samples with discrepant HC2/Onclarity results. Conversely, 88.2% of 51 HC2−/BD− samples from controls were also negative by H13.
Agreement of H13 results with a combination of HC2 and Onclarity assay (BD) results, overall and by disease status in the study populationa
Considering the 15 samples positive by the virologic standard (HC2+/BD+) but negative by H13, 10 (66.6%) were positive for HPV types 56 (HPV56), 59, and/or 66 (56/59/66) by Onclarity; thus, negativity was possibly partly explained because H13 lacks cross-reactivity with HPV66 (Table 3). In this set of specimens (HC2+/BD+ and H13−), the H13 assay did not detect one specimen with the genotype HPV16 (classified histologically as CIN3 but cytologically as normal), one HPV18 (CIN2 and atypical squamous cells of undetermined significance [ASC-US]), one HPV31 (CIN2 and ASC-US), and one HPV51 (normal histology and ASC-US).
Detailed results of the discordant specimens between the virologic standard (HC2+/BD+) and H13 (negative)
When the detection rates of HPV between the H13 and HC2 assays were compared by HPV genotype categories, the positivity rates were comparable across all categories, except for the category including HPV genotypes 56, 59, and 66 (corresponding to the HPV56/59/66 channel of the Onclarity assay), where HC2 detected 10 more cases of HPV than H13 (Table 4).
HPV genotyping by Onclarity in relation to H13 or HC2 assay statusa
DISCUSSION
Main findings.In this study, we assessed the concordance of a new lower-cost real-time PCR HPV DNA detection assay (H13) with two validated assays (HC2 and Onclarity) and compared the performance of H13 with that of the reference HPV tests in detecting high-grade cervical lesions (CIN2+). The agreement between H13 and a combination of HC2 and Onclarity was found to be good, since H13 correctly identified 94.4% of HPV-positive (HC2+/BD+) samples among cases (CIN2+) and correctly identified 88.2% of HPV-negative (HC2−/BD−) samples among controls (<CIN2). Moreover, H13 appears to have less cross-reaction with untargeted HPV types than HC2. To the best of our knowledge, this is the first study in the international literature that assesses analytical and clinical performance of the H13 assay for cervical cancer screening in comparison with two other validated assays. The samples used in this study included a wide range of participant ages and cytologic and histologic abnormalities to allow the assay to be assessed on a broad scope of clinical conditions.
Comparison to the literature.In limited-resource settings, where up to 85% of new cases of cervical cancer and 87% of deaths related to this preventable disease occur (2), HPV testing is a promising screening tool that could plausibly be implemented on a large scale. The H13 assay is relatively easy to perform and requires only modest laboratory skills, and results can be provided within a few hours. In addition, this HPV assay is likely to perform like other molecular assays, with the advantage of being more accurate and more reliable than VIA or cytology (3, 4). Indeed, the positivity rate of H13 among women with high-grade cervical lesions (89.9%) in this study is comparable to the sensitivity of careHPV to detect CIN2+ (85.7%) among asymptomatic sexually active women eligible for cervical screening (9).
The decision to introduce cost-effective HPV-based screening programs in limited-resource settings should take into account, in addition to sensitivity and reliability, the capacity of the HPV assay to properly identify women without high-grade cervical lesions, to avoid overburdening limited human and treatment resources. In this study, the estimated positivity rate of assays among controls could be used as an unweighted proxy for their false-positive rates (equal to 1 − specificity). H13 showed a positivity rate (53.5%) significantly lower than that of HC2 (72.4%, P < 0.001) among controls. This result is consistent with that of Cheng et al. in China (8), who found that the positivity rates among women with <CIN2 was lower for H13 (63.7%) than for HC2 (68.2%) and that the resulting specificity in detecting CIN2+ was slightly higher for H13 (81.1%) than for HC2 (79.8%), although that difference was not statistically significant (P = 0.82). This could be explained by the fact that in comparison to HC2, the H13 assay that targets the same high-risk HPV types as HC2 appears to be less cross-reactive with untargeted HPV types and thus can better identify women without CIN2+ than HC2. Since the careHPV assay is based on the HC2 technology, H13 might also be less cross-reactive with untargeted HPV types than careHPV.
The cost of H13, in an ongoing cervical cancer screening program in Latin America and West Africa, was estimated at about US$6 per test, approaching affordability in some low-resource settings. In this respect, H13 is competitive with other HPV assays that are designed for settings with limited resources, such as careHPV.
Limitations.The present study had some limitations. First, our study sample consisted predominantly of HPV-positive specimens and thus was not representative of any real population. This is the reason why sensitivity and specificity, as well as McNemar's test and Cohen's kappa statistic, could not be estimated. However, we used the positivity rate of either assay in relation to disease status as a measure of clinical performance. We also used the absolute agreement between either pair of tests as a measure of concordance among assays. Another limitation of the study was the proportion of samples with invalid H13 results (143/516, 27.7%) as determined by the negative internal control of DNA adequacy. This is unlikely to be related to intrinsic problems with the assay, since in the study conducted in China, no invalid result was reported with H13 (8), and since in ongoing analyses with freshly collected samples at the U.S. National Cancer Institute (NCI), the rate of invalid results of H13 is under 3%. Moreover, this high proportion was expected at the onset of the study, since the residual samples used to perform H13 analyses have been used for multiple analyses and stored for many years, to such an extent that some of the remaining specimens had insufficient or degraded DNA for H13 analysis. However, the comparable distributions of variables among samples with valid H13 results and those with invalid H13 results suggest that removal of invalid H13 results from our analyses did not significantly affect our results.
Conclusion.H13 corresponds well to the combination of HC2 and Onclarity and has good clinical accuracy compared to histologic diagnosis, with less cross-reactivity than that of HC2. Given its low cost and ease of use, H13 might be useful for primary screening in limited-resource settings. Further confirmatory studies are needed to evaluate the analytical and clinical performance of H13 on representative samples from populations in limited-resource settings.
MATERIALS AND METHODS
Study population.We evaluated H13 using a convenience sample of 516 residual exfoliative cervical specimens chosen from the Kaiser Permanente Northern California (KPNC) and NCI repository of discarded cervical specimens that remained following routine HPV testing. The biorepository is the basis of the Persistence and Progression (PaP) cohort study, an ongoing study of approximately 55,000 women who obtained cervical screening at KPNC. KPNC introduced cytology (Pap smear) and HPV cotesting with Hybrid Capture 2 (HC2; Qiagen, Germantown, MD) at 3-year intervals in 2003 (10). The PaP study, starting in 2007, stored residual exfoliative cervical specimens remaining from HC2 testing until 2011 (11). Since the goal of the PaP cohort is molecular epidemiologic studies of risk factors for precancer in HPV-infected women, approximately 45,000 (>80%) of the samples included were chosen from women who were HC2+ at enrollment into the cohort. About 10,000 residual specimens from HC2− women were collected for comparison. Original HC2 results, Onclarity retest results, and clinical data including cervical cytology and histology results were obtained from electronic records of participants who chose not to opt out of use of the waste specimens. The PaP cohort has been described in detail in other publications (11, 12).
A convenience sample of 516 specimens from women in the PaP cohort who underwent routine screening (as opposed to being tested for previous cervical abnormalities) and were tested for HPV typing with the Onclarity assay (12) was chosen for use in the present study. In this group of women, we performed a stratified random sampling of specimens to be used in this study. Stratification was done on the following potential confounding factors: histology results (classified into two categories: CIN2+; <CIN2), cytology results (classified into three categories: [i] negative for intraepithelial lesion or malignancy [NILM]; [ii] atypical squamous cells of undetermined significance [ASC-US] and low-grade squamous intraepithelial lesions [LSIL]; [iii] atypical squamous cells not excluding HSIL [ASC-H], atypical glandular cells [AGC], adenocarcinoma in situ [AIS], high-grade squamous intraepithelial lesions [HSIL], squamous cell carcinoma [SCC], and adenocarcinoma), and age (classified into three categories: <45 years; 45 to 54 years; >54 years). Those variables allowed us to define 12 strata, in each of which specimens were randomly selected. Since HC2-positive women were oversampled in the PaP cohort study, this subset of the PaP collection also contains predominantly HC2-positive women.
HC2.Hybrid Capture 2 (HC2) is an in vitro nucleic acid hybridization assay with signal amplification using microplate chemiluminescence for the qualitative detection of HPV DNA in cervical specimens. It targets, as a pool, the major carcinogenic HPV types, namely, HPV16, -18, -31, -33, -35, -39, -45, -51, -52, -56, -58, -59, and -68. In addition, the HC2 probe cross-reacts with other human Alphapapillomavirus types, including types of uncertain or unknown carcinogenicity (e.g., HPV11, -53, -61, -66, -70, and -81) (13). This assay was the first HPV test approved by the U.S. Food and Drug Administration (FDA) for cervical cancer screening and is available in most settings. HC2 tests were performed for clinical purposes within 7 days of specimen collection using specimens collected in specimen transport medium (STM; Qiagen) according to the manufacturer's instructions. The residual alkalinized cervical specimens in STM were neutralized with acid and stored frozen for future testing (12).
Onclarity.The Onclarity assay (BD) is a multiplex qPCR assay that targets HPV type-specific E6 or E7 sequences (10). The assay provides genotyping information for 14 HPV types and utilizes 9 channels for (i) types 16, 18, 31, 45, 51, and 52 (separately), (ii) types 33 and 58 (HPV33/58) (pooled), (iii) types 35 and 68 (HPV35/68) (pooled), and (iv) types 56, 59, and 66 (HPV56/59/66) (pooled). An FDA trial for U.S. licensure is still under way, but the test has obtained CE (European Conformity) marking and is marketed in Europe. DNA extraction, PCR amplification, and detection were performed on stored specimens on the BD Viper LT system in accordance with the manufacturer's instructions. Laboratory scientists who carried out HPV testing with Onclarity were masked to all other data, and the Onclarity results derived from the nine typing channels were analyzed and interpreted independently. Onclarity analyses were performed at the NCI between February and December 2011.
Hybribio H13.The Hybribio qPCR kit (H13) detects and produces a pooled result for 13 high-risk HPV types (types 16, 18, 31, 33, 35, 39, 45, 51, 52, 56, 58, 59, and 68), using viral type-specific probes in the E6 gene. H13 has two labeled probes, one that serves as internal control for DNA adequacy (by detecting human beta-globin) and the other for the detection of HPV DNA. Each probe consists of an oligonucleotide labeled with a reporter fluorophore at the 5′ end and a quencher fluorophore at the 3′ end. During the PCR, the probe anneals specifically between the forward and reverse primers to an internal region of the PCR product. The real-time instrument used was a Roche LightCycler. The extension of the primer and replication of the template to which the probe is bound are carried out by Taq polymerase. The 5′ exonuclease activity of the Taq polymerase cleaves the probe, and the reporter fluorophore is released away from the close vicinity of the quencher. The fluorescence intensity of the reporter dye increases each PCR cycle. In order to detect fluorescence, both primers and probes have to be specific to the same template. Therefore, nonspecific amplification is not detected. In this study, DNA was extracted from STM that had been processed for Onclarity retesting, by using a novel silica bead genomic DNA extraction method (SiliEx), currently being developed at the NCI Laboratory for Translational Genomics. The method uses magnetic beads coated with silica to bind to DNA from 100 μl of STM. One microliter of the resulting DNA template was used for each PCR tube. The procedure is different from that described in the manufacturer's instructions on extraction of DNA and DNA template quantity. H13 testing was conducted from January 2017 to February 2017. Laboratory scientists who performed the H13 analysis at the NCI and who are familiar with other HPV assays (HC2, BD Onclarity) consistently reported that H13 was easier to perform.
When the internal control was positive, H13 results were classified as positive (H13+) when at least one of the 13 high-risk HPV types was detected or negative (H13−) otherwise. When the internal control was negative, the H13 test result was considered not valid.
Cytology.Until 2009, KPNC used conventional Pap smears and slides were reviewed manually with assistance from the BD FocalPoint slide profiler (BD Diagnostics, Burlington, NC) (10, 11). Since 2009, KPNC has replaced conventional Pap smears with liquid-based cytology (SurePath; BD Diagnostics, Sparks, MD). Cytology results were reported according to the 2001 Bethesda system, and for the purpose of this study, they were grouped into three categories: (i) normal; (ii) minor abnormality, including ASC-US and LSIL; and (iii) higher-grade abnormality, including ASC-H, AGC, HSIL, AIS, SCC, and adenocarcinoma.
Histology.Women in the KPNC PaP cohort were referred to colposcopy and biopsy if indicated, based on HC2 and cytology cotesting results according to standard clinical practice (10). The worst histology results observed during follow up post-specimen collection for each woman were considered in the analysis. Women not referred for colposcopy or who did not have histology results under the screening algorithm were considered to have no precancer. Histology results were divided into two categories: (i) CIN2+ or cases, and (ii) <CIN2 or controls. Women with uninterpretable biopsy results were excluded from our analyses.
Statistical analysis. (i) Characteristics of specimens with valid versus invalid results.Since H13 testing for this study was based on small amounts of residual aliquots of cervical specimens that remained after use for other analyses, the number of invalid results among the samples tested was high, as anticipated (143/516, 27.7%). To assess the possible influence of this subset of samples on our study's results, we described the study population by comparing demographic and biological features between samples with and without valid H13 results. Continuous variables were described as medians with interquartile ranges (IQR), and categorical variables were described as percentages with 95% confidence intervals (95% CI). For comparisons, a P value of <0.05 was considered statistically significant.
(ii) H13 test performance.Since specimens used in this study were chosen as a convenience sample of women with a high prevalence of HPV infection and abnormal histopathology, the kappa statistic and McNemar's chi-square test were considered biased in this nonrepresentative population. Recognizing the effect of sampling on agreement statistics, we did not attempt to reconstitute the sampling fractions owing to the high number of invalid results by the H13 assay. Rather, we calculated H13 positivity relative to a mixed viral/histologic standard for the 373 samples with valid H13 results, i.e., those specimens that tested positive or negative on both/either/neither HPV reference test, stratified by disease status.
Specifically, we analyzed the performance of H13 by combining two reference standards, one histologic and one virologic. The virologic reference standard was based on the combination of HPV testing results for HC2 and Onclarity, which were available for all specimens. This enabled us to define four categories: samples positive by both HC2 and Onclarity (HC2+/BD+), samples positive by HC2 and negative by Onclarity (HC2+/BD−), samples negative by HC2 and positive by Onclarity (HC2−/BD+), and samples negative by both HC2 and Onclarity (HC2−/BD−). The histologic reference used the worst histology results available to define two levels of disease status (<CIN2 and CIN2+). We computed the positivity rate of the H13 test (percentage of samples tested positive by H13) by the eight-category combination of virologic and histopathologic standards.
To assess the possibility and extent of cross-reactivity of H13 with untargeted HPV types, we compared HPV positivity rates between the H13 and HC2 assays, by HPV genotype categories obtained with the Onclarity assay. All analyses were conducted using STATA 14 software (StataCorp LLC, Texas, USA).
Ethical considerations.H13 testing was conducted at the National Cancer Institute's Laboratory of Translational Genomics. Personal identifiers were removed from specimens, and study identifiers that could be linked to clinical data but not directly to personal identifiers were assigned. Those involved in the study did not have access to personally identifying information. Both KPNC and NCI IRB review committees have approved the PaP cohort study and reapproved it yearly.
ACKNOWLEDGMENTS
The Visiting Fellowship position of Joel Fokom Domgue at the Division of Cancer Epidemiology and Genetics (DCEG) of the U.S. National Cancer Institute (NCI) is sponsored by the NCI's Centre for Global Health (CGH). However, the NCI's CGH is not directly involved in this research.
The H13 kits used in this study were purchased at their usual quoted price by the Laboratory of Translational Genomics, DCEG, NCI, headed by Michael Dean, via its regular intramural budget. However, this research received no support from Hybribio, which developed the H13 assay.
Although the funder (NCI is one of the National Institutes of Health [NIH], a U.S. federal agency) employs some of the scientists involved in this study, NCI/NIH had no role in the study design, data collection and interpretation, or decision to submit the work for publication.
We declare no conflicts of interest.
FOOTNOTES
- Received 23 March 2017.
- Returned for modification 10 April 2017.
- Accepted 8 May 2017.
- Accepted manuscript posted online 17 May 2017.
- Copyright © 2017 American Society for Microbiology.
