This Article
Right arrow Abstract Freely available
Right arrow Full Text (PDF)
Right arrow Alert me when this article is cited
Right arrow Alert me if a correction is posted
Services
Right arrow Similar articles in this journal
Right arrow Similar articles in PubMed
Right arrow Alert me to new issues of the journal
Right arrow Download to citation manager
Right arrowReprints and Permissions
Right arrow Copyright Information
Right arrow Books from ASM Press
Right arrow MicrobeWorld
Citing Articles
Right arrow Citing Articles via HighWire
Right arrow Citing Articles via Google Scholar
Google Scholar
Right arrow Articles by Kornegay, J. R.
Right arrow Articles by Coutlée, F.
Right arrow Search for Related Content
PubMed
Right arrow PubMed Citation
Right arrow Articles by Kornegay, J. R.
Right arrow Articles by Coutlée, F.

 Previous Article  |  Next Article 

Journal of Clinical Microbiology, March 2003, p. 1080-1086, Vol. 41, No. 3
0095-1137/03/$08.00+0     DOI: 10.1128/JCM.41.3.1080-1086.2003
Copyright © 2003, American Society for Microbiology. All Rights Reserved.

International Proficiency Study of a Consensus L1 PCR Assay for the Detection and Typing of Human Papillomavirus DNA: Evaluation of Accuracy and Intralaboratory and Interlaboratory Agreement

Janet R. Kornegay,1 Michel Roger,2,3 Philip O. Davies,4 Amanda P. Shepard,1 Nayana A. Guerrero,5 Belen Lloveras,5 Darren Evans,4 and François Coutlée1,2,3*

Roche Molecular Systems Inc., Alameda, California,1 Département de Microbiologie et Infectiologie, Hôpital Notre-Dame du Centre Hospitalier de l'Université de Montréal,2 Département de Microbiologie et Immunologie, Université de Montréal, Montréal, Québec, Canada;,3 BMI, London, United Kingdom,4 Department of Pathology, CSU Bellvitge/Lab. Recerca Translacional, Institut Catala d'Oncologia, Barcelona, Spain5

Received 24 May 2002/ Returned for modification 26 August 2002/ Accepted 9 December 2002


arrow
ABSTRACT
 
The PGMY L1 consensus primer pair combined with the line blot assay allows the detection of 27 genital human papillomavirus (HPV) genotypes. We conducted an intralaboratory and interlaboratory agreement study to assess the accuracy and reproducibility of PCR for HPV DNA detection and typing using the PGMY primers and typing amplicons with the line blot (PGMY-LB) assay. A test panel of 109 samples consisting of 29 HPV-negative (10 buffer controls and 19 genital samples) and 80 HPV-positive samples (60 genital samples and 20 controls with small or large amounts of HPV DNA plasmids) were tested blindly in triplicate by three laboratories. Intralaboratory agreement ranged from 86 to 98% for HPV DNA detection. PGMY-LB assay results for samples with a low copy number of HPV DNA were less reproducible. The rate of intralaboratory agreement excluding negative results for HPV typing ranged from 78 to 96%. Interlaboratory reliability for HPV DNA positivity and HPV typing was very good, with levels of agreement of >95% and kappa values of >0.87. Again, low-copy-number samples were more prone to generating discrepant results. The accuracy varied from 91 to 100% for HPV DNA positivity and from 90 to 100% for HPV typing. HPV testing can thus be accomplished reliably with PCR by using a standardized written protocol and quality-controlled reagents. The use of validated HPV DNA detection and typing assays demonstrating excellent interlaboratory agreement will allow investigators to better compare results between epidemiological studies.


arrow
INTRODUCTION
 
Human papillomavirus (HPV) infection is a strong independent predictor for squamous intraepithelial lesions (SIL) and invasive cancer of the uterine cervix (10, 34). Women with persistent HPV infection are at highest risk for development of cervical SIL or evolution from low-grade SIL to higher-grade disease of the uterine cervix (8, 24). The causal association between HPV and cervical cancer was demonstrated by epidemiological investigations that used PCR, the most sensitive tool for HPV detection and typing, in cells collected from the uterine cervix (5).

Because of the genetic polymorphism of HPV, consensus PCR assays have been utilized to amplify in one reaction the majority of known, as well as novel, anogenital HPV genotypes. The most widely used primer sets, the MY09/MY11/HMB01, GP5+/GP6+, PGMY09/PGMY11, and SPF1/SPF2 consensus primers, target conserved sequences in the HPV L1 gene (1, 7, 12, 15, 18, 20, 23). To render the consensus PCR assays more feasible for large-scale testing and clinical application, convenient assays for the detection and typing of HPV have been developed for all primer sets. HPV amplicons generated by PGMY primers can be easily detected and typed by a nonisotopic reverse hybridization assay, the line blot (LB) assay (4, 13). Recently, a colorimetric microtiter plate-based enzyme immunoassay was also reported for screening of the broad spectrum of HPV amplified by the PGMY primers using a generic probe mix (21).

The proficiency of microbiology laboratory testing is generally monitored by proficiency-testing programs that also allow the determination of test variability between laboratories (32). Implementation of proficiency testing panels is essential for unregulated molecular diagnostic tests. Proficiency panels have been developed for the molecular diagnosis of several infectious agents (25, 32) and in research settings to monitor the performance of molecular virology laboratories (17). Thus far, no proficiency panel has been constructed and made available to the general research community for HPV testing. The validity of HPV DNA detection and typing with PCR assays has not been thoroughly assessed.

Epidemiological studies and vaccine clinical trials require the reliable and reproducible identification of genital HPV infection. Several studies have evaluated the intermethod variation of HPV DNA detection (2, 11, 14, 19, 22, 26, 28, 30, 33). However, few studies have evaluated the intralaboratory and interlaboratory reproducibility of L1 consensus PCR assays although these assays have been widely used (6, 16, 19). The latter studies were conducted with in-house reagents and protocols. Although the consensus L1 PCR assays are not commercially available, standardized reagents and protocols for the PGMY-LB assay are available from Roche Molecular Systems for research purposes. The use of standardized and reproducible protocols for HPV detection and typing could facilitate the comparison of results between studies on HPV infection.

In order to assess the accuracy and reproducibility of the PGMY-LB assay, three laboratories were invited to participate in a collaborative study to compare their abilities to detect the presence of and to genotype HPV DNA in blinded specimens. We report here the estimate of intralaboratory and interlaboratory reproducibility of the PGMY-LB assay for HPV DNA detection and typing on 109 specimens tested in triplicate by three independent laboratories under standardized conditions. This information is useful in view of the wider use of the PGMY-LB assay by numerous research groups, as well as its potential application in diagnostic laboratories for HPV detection and typing.


arrow
MATERIALS AND METHODS
 
Selection of samples for the test panel. Roche Molecular Systems established the test panel of samples with the PGMY-LB assay. The quality control set included 30 synthetic samples: negative buffer samples devoid of HPV DNA sequences (n = 10), samples containing 0.063 µg of human genomic DNA per 5 µl (catalog no. 1691112; Roche Molecular Biochemicals, Indianapolis, Ind.) spiked with 4,000 copies of HPV-16 DNA plasmid per 5 µl (n = 5), human DNA spiked with 400 copies of HPV-16 DNA plasmid per 5 µl (n = 5), human DNA spiked with 4,000 copies of HPV-45 DNA plasmid per 5 µl (n = 5), and human DNA spiked with 400 copies of HPV-45 DNA plasmid per 5 µl (n = 5). The panel also included 79 anonymous cervical samples collected with cytobrushes from 79 women attending clinics at the BMI hospitals in London. The latter samples came from a set of 90 samples that had been screened initially with the PGMY-LB assay at the BMI Health Services Laboratory and stored frozen. Roche Molecular Systems retested with the PGMY-LB assay the 90 clinical samples in duplicate to ensure the presence of HPV genotypes detected initially. Confirmation of initial PCR results was not obtained for 11 of the 90 samples (8 generated discordant results between PCR runs, 2 had been mislabeled, and 1 was only tested once at Roche), leaving 79 clinical samples (19 HPV negative and 60 HPV positive) in the panel.

Of the 109 samples included in the proficiency-testing panel, 29 were HPV-negative specimens (10 buffer controls and 19 genital samples) and 80 samples contained HPV DNA. The samples containing HPV DNA included 47 genital samples with one HPV type, 10 synthetic samples with low HPV DNA copy numbers, 10 synthetic samples with high HPV DNA copy numbers, 3 synthetic samples containing multiple HPV types, and 10 genital samples containing, in addition to the genotype(s) consistently detected in repeated PGMY-LB assay runs, at least one type that was not consistently detected. The latter 10 samples were tested four times by Roche Molecular Systems: an additional type was detected in one out of four runs (two samples), two out of four runs (seven samples), or three out of four runs (two samples). One sample contained two HPV types that were inconsistently detected in the four runs. HPV genotypes that were inconsistently detected in these 10 samples included types 42 (in three samples), 31 and 58 (in two samples each), and 16, 35, 33, and 84 (in one sample each).

In addition to the 10 samples containing HPV-16 plasmids and 10 samples containing HPV-45 plasmids, there were a variety of HPV genotypes represented among the 60 HPV-positive samples. They included types 16 (in 10 samples), 18 and 51 (in 5 samples each), 31, 45, 52, 59, and 66 (in 4 samples each), 33, 39, 42, and 54 (in 3 samples each), 56, 68, 83, and 6 (in 2 samples each), and 35, 53, 58, 82, 84, and 73 (in 1 sample each). Of the three samples with multiple types, one contained types 16 and 31, one contained types 16, 18, 52, and 73, and another one contained types 16, 35, 42, and 82. The results from the detection of ß-globin were not considered and compared in this work.

Study design. Three laboratories with different levels of experience with molecular diagnostic methods participated in the study. Laboratory A had experience in human genetic testing but was unfamiliar with molecular virology tests. Laboratory B was a diagnostic molecular microbiology laboratory specifically trained to perform the PGMY-LB assay. Laboratory C was a diagnostic cytopathology laboratory familiar with commercialized molecular diagnostic techniques but unfamiliar with the PGMY-LB assay. Standardized reagents comprising the PCR master mix, LB strips, and reagents were sent to each participating center along with a written standard operating procedure for the PGMY-LB assay. Each participating center was asked to follow the protocol without modification.

Three 20-µl aliquots from each panel sample were coded and distributed on dry ice by Roche Molecular Systems to each of the three independent laboratories. In the test panel, the HPV-negative or buffer controls were distributed randomly among the HPV-positive specimens. Five microliters of each sample was used for HPV testing with the PGMY-LB assay. Each laboratory tested, in different PCR runs, each of the 109 samples three times, blinded to the results obtained from Roche Molecular Systems, from the other laboratories, and from previous runs. Laboratories A and B also tested PGMY amplicons with the generic probe microplate assay. The study testing was conducted between June and August 2000.

PGMY-LB assay. HPV DNA was amplified in each center under standard conditions with the L1 consensus HPV PGMY09/PGMY11 primer set, as previously described (12). The amplification mixture contained 4 mM MgCl2, 50 mM KCl, 7.5 U of AmpliTaq Gold DNA polymerase (Applied Biosystems), 200 µM concentrations each of dATP, dCTP, and dGTP, 600 µM dUTP, 100 pmol of each biotinylated PGMY primer pool, and 5 pmol each of the 5'-biotinylated ß-globin primers GH20 and PC04. HPV was amplified with the ultrasensitive profile that consisted of activation of AmpliTaq Gold at 95°C for 9 min, denaturation for 1 min at 95°C, annealing for 1 min at 55°C, and extension at 72°C for 1 min for a total of 40 cycles. Amplification was followed by a 5-min terminal extension step at 72°C. Laboratory A used a Biometra Uno II thermocycler, laboratory B used a TC 9600 thermocycler, and laboratory C used an MJ Research PTC-1 thermocycler. Measures to avoid false-positive reactions due to contamination were strictly adhered to. HPV genotyping was performed with the reverse LB detection system as previously described (13). PCR products were denatured in 0.4 N NaOH and hybridized to an immobilized probe array containing probes for 27 HPV genotypes (types 6, 11, 16, 18, 26, 31, 33, 35, 39, 40, 42, 45, 51, 52, 53, 54, 55, 56, 57, 58, 59, 66, 68, 73, 82, 83, and 84). Positive hybridization was detected by streptavidin-horseradish peroxidase-mediated color precipitation on the membrane at the probe line.

Generic probe microplate assay. Generic probe detection reactions were performed by using reagents from a PCR-ELISA DIG detection kit (Roche Molecular Biochemicals) as described previously (21). Twenty microliters of denatured PCR products was added to the streptavidin-coated microtiter wells, followed by the addition of 200 µl of hybridization buffer provided by Roche and 20 µl of the denatured generic probe pool. Roche synthesized the digoxigenin-labeled generic probe pool by amplification of DNA from HPV types 11, 16, 18, and 51 as described previously (21). Hybridization was performed at 37°C for 1 h. Following color development, absorbance was measured at 405 nm and the background, defined as the average value of blank cells containing no PCR product, was subtracted from all values. A specimen was considered positive if the corrected A405 was greater than 0.5, negative if the value was less than 0.2, and indeterminate in the range between 0.2 and 0.499.

Statistical methods. To ensure an independent evaluation of the reproducibility of the assay, statistical analyses were performed by a scientist (F.C.) without financial interests in Roche Molecular Systems. Results from the three laboratories were imported into a common database for comparison (Microsoft Excel). Results obtained by Roche Molecular Systems (the reference laboratory) were considered the "gold standard" (HPV DNA reference standard). Agreement for overall HPV positivity (HPV DNA positive irrespective of types identified) and for type-specific positivity was calculated as the percentage of runs with identical results for the presence or absence of HPV. Cohen's unweighted kappa statistic was calculated to adjust for chance agreement between sites or between sites and HPV DNA reference standard results (9). In general, a kappa value of >0.75 indicates excellent agreement beyond chance, a kappa value between 0.40 and 0.75 indicates fair to good agreement, and a kappa value of <0.40 represents poor agreement.

The reproducibility of repeated PCR assays (intralaboratory reproducibility) was calculated for each site by comparing results from triplicate runs and calculating the crude percent agreement and the percent agreement considering only HPV-positive results. The intralaboratory reproducibility for the presence or absence of each of 27 genotypes was first calculated by using crude typing results. Since 10 samples contained HPV types that were not consistently detected between runs by the reference laboratory, agreement was recalculated by not considering the presence or absence of these additional types (see above).

Interlaboratory agreement (interlaboratory reproducibility) was assessed by pairwise comparisons of test results from the three laboratories calculating the crude percent agreement and the kappa statistic. Results obtained by the reference laboratory were not considered in these comparisons. To avoid combining intralaboratory and interlaboratory variability in the comparisons of the three laboratories, two strategies of analysis were considered. Laboratories were first compared by considering the HPV DNA and typing results obtained on the initial run for each sample in each laboratory. This strategy allows evaluation of interlaboratory reproducibility when samples are only tested once, as is often the case. Laboratories were also compared by using a consensus definition of HPV results: the final HPV result for a specimen was that obtained from at least two out of three runs for the presence of HPV DNA and HPV typing. This definition favors a higher degree of concordance between laboratories but takes into account triplicate results for each sample.

Accuracy of HPV DNA detection and HPV typing was assessed by pairwise comparisons in contingency tables of each laboratory with the HPV DNA reference standard results, using the two-sided McNemar chi-square analysis for matched-pair data. Agreement was determined first by considering results of the first aliquot for each sample and then by considering the consensus HPV results for each sample as explained above. Since the reference laboratory did not consistently detect a type in 10 samples, agreement was recalculated by not considering the presence or absence of these additional types as a discordant result. Results from laboratories A and B for the generic probe microplate assay were compared for HPV DNA positivity with results obtained with the LB assay. Indeterminate results were excluded from the calculation of agreement.


arrow
RESULTS
 
Results from triplicate runs with the PGMY-LB assay were compared in each laboratory to first assess the reproducibility of HPV DNA detection irrespective of the type(s) detected. The intralaboratory agreement for HPV DNA detection of the PGMY-LB assay was high for each site (Table 1). The agreement between triplicates for each center was lower when only the HPV-positive results were considered. Two laboratories had intralaboratory levels of agreement of >97% for HPV DNA detection when all results were considered. A significant difference was found in the number of discordant triplicates for HPV DNA detection among the three participating laboratories (Table 1): 15 (13.8%), 3 (2.8%), and 2 (1.8%) of 109 samples generated discordant triplicates for HPV DNA in laboratories A, B, and C, respectively (P = 0.005; Pearson chi-square test). When laboratories were compared two by two, significant differences were found, after the Bonferroni correction for a total of three comparisons, in the proportion of discordant triplicates between laboratories A and B and between laboratories A and C (P = 0.009 and 0.003, respectively; Pearson chi-square test). The difference in the number of discordant triplicates between laboratories B and C did not reach statistical significance (P = 0.65; Pearson chi-square test).


View this table:
[in this window]
[in a new window]
  		TABLE 1. Intralaboratory reproducibility of PGMY-LB for HPV DNA detection and HPV typing on triplicate testing of 109 samples by three laboratories

An intralaboratory agreement analysis stratified by the amount of HPV DNA introduced in the sample for laboratory A that had the highest number of discordant triplicates revealed that the presence of a low copy number of HPV DNA had an influence on reproducibility. In this laboratory, 7 (70.0%) of the 10 samples containing small amounts of HPV DNA generated discordant triplicates for HPV DNA detection, as opposed to 8 of the 70 samples with clear signals in the PGMY-LB assay or high copy numbers (P = 0.002; Fisher's exact test).

In order to estimate the intralaboratory agreement for HPV DNA typing, we compared the results of triplicate testing of 109 samples for 27 genotypes (2,943 type-specific results per center) (Table 1). Although the number of discordant triplicates was greater for typing results than HPV DNA detection in all laboratories, agreement reached at least 99% when all specimens were considered (Table 1). The high number of HPV type-specific negative results explains the greater intralaboratory agreement obtained with HPV typing results than with generic HPV DNA detection despite a greater number of discordant triplicates. When concordant negative triplicates were excluded from the analysis, the level of agreement decreased as shown in Table 1. We then considered in our evaluation the fact that 10 samples contained HPV types that were not consistently detected between runs by the reference laboratory. In laboratory C, 7 (50%) of the 14 discordant triplicates contained HPV types not uniformly detected by the reference laboratory, thus leaving only 7 truly discordant triplicates for HPV typing (values in parentheses in Table 1). In laboratory B, four (50%) of the eight discordant triplicates contained HPV types not uniformly detected by the reference laboratory, leaving only four truly discordant triplicates for HPV typing. In laboratory A, 4 (17.3%) of the 23 discordant triplicates contained HPV types not uniformly detected by the reference laboratory, leaving 19 truly discordant triplicates for HPV typing. When only truly discordant triplicates were considered, the intralaboratory agreement for HPV typing increased slightly when all results were included in the calculation of agreement and ranged from 78.4 to 95.8% when only HPV-positive results were used (Table 1).

When the numbers of truly discordant triplicates for HPV DNA typing were compared between participating sites, significant differences were found, after the Bonferroni correction for a total of three comparisons, between laboratories A and B and between laboratories A and C (P = 0.006 and 0.054, respectively; Pearson chi-square test). Considering typing results from laboratory A that showed the greatest intralaboratory variability, 8 of 10 samples with low copy numbers of HPV DNA generated discordant triplicates, as opposed to 15 of 70 samples with clear signals or high HPV copy numbers (P = 0.005; Fisher's exact test). This relationship was not found for laboratories B and C, which had a higher level of agreement between triplicates (data not shown). Excluding specimens with low copy numbers of HPV types 45 and 16, discordant triplicates were distributed equally across several types. Reproducibility was perfect for the three samples with multiple HPV type infections for two laboratories (data not shown). One laboratory failed to detect an HPV type in one of the triplicates for two of the latter samples (data not shown).

Interlaboratory reproducibility of the PGMY-LB assay was first assessed by considering only the first PCR run in each laboratory. Laboratory pairwise comparisons are shown in Table 2 for HPV DNA detection and typing. Agreement for HPV DNA positivity between laboratory pairs A and B, B and C, and A and C reached 94% (102 of 109 samples, kappa value of 0.85), 99% (108 of 109 samples, kappa value of 0.98), and 93% (101 of 109 samples, kappa value of 0.83), respectively. Agreement for HPV typing results between laboratory pairs A and B, B and C, and A and C reached 94% (102 of 109 samples, kappa value of 0.81), 97% (106 of 109 samples, kappa value of 0.88), and 93% (101 of 109 samples, kappa value of 0.80), respectively.


View this table:
[in this window]
[in a new window]
  		TABLE 2. Interlaboratory agreement of PGMY-LB results for HPV DNA detection and HPV typing obtained on initial testing of 109 samples by three laboratories

Interlaboratory reproducibility of the PGMY-LB assay was then assessed by using a consensus definition of HPV positivity (two of three runs with identical results), as explained in Materials and Methods. Laboratory pairwise comparisons are shown in Table 3 for HPV DNA detection and typing. Interlaboratory reliability for HPV DNA positivity and HPV typing was excellent, with levels of agreement of >95% and kappa values of >0.87, including perfect kappa values for HPV DNA detection and HPV typing between laboratories B and C. Of the four samples testing negative in laboratory A but positive in the two other laboratories, two contained low copy numbers of HPV-16 or -45. Low-copy-number samples were more prone to generate discordant results in laboratory A, since 2 (20%) of 10 low-copy-number samples versus 2 (3%) of 70 other HPV-positive samples were misclassified by laboratory A (P = 0.07; Fisher's exact test).


View this table:
[in this window]
[in a new window]
  		TABLE 3. Interlaboratory agreement of PGMY-LB results for HPV DNA detection and HPV typing obtained in three laboratories on triplicate testings of 109 samples using the consensus definition for HPV results

The accuracy of the PGMY-LB assay was measured by comparing results obtained from each laboratory with those obtained at Roche Molecular Systems (HPV DNA reference standard) for 109 samples. Additional HPV types inconsistently detected by Roche in multiple testing in 10 samples were not considered in this analysis. For the 29 HPV-negative samples, only 2 (0.8%) of 261 assays scored positive for HPV DNA. One specimen that tested falsely positive for HPV-51 was preceded by an HPV-51-positive sample, while HPV-31 was responsible for the other false-positive result. Since samples were tested in triplicate, results from the first run and from a consensus definition of HPV positivity were compared separately to the HPV DNA reference standard results, as shown in Table 4. The percentages of agreement between each laboratory and the HPV DNA reference standard varied from 91 to 100% for HPV DNA positivity and from 90 to 100% for HPV typing. Laboratory B correctly identified all HPV-positive samples using results from the first assay or the consensus definition of HPV positivity. HPV types were identified correctly in all samples when the consensus definition of HPV positivity was used and in 79 (98.8%) samples when the first run only was considered. Similarly, laboratory C correctly identified 99% of HPV-positive samples and correctly identified HPV types for nearly all samples except three (3.8%) and one (1.3%) when the first run and the consensus definition of HPV positivity were used, respectively. When the first sample was considered, laboratory A failed to identify 7 (9%) of 80 HPV-positive samples and failed in HPV typing for 8 (10%) for 80 HPV-positive samples. Four (57.1%) of the seven HPV-positive samples that were misclassified by laboratory A in the first run contained low copy numbers of HPV DNA (data not shown).


View this table:
[in this window]
[in a new window]
  		TABLE 4. Accuracy of HPV DNA detection and typing results obtained with PGMY-LB for 109 samples by three laboratories

The agreement for HPV-positive samples, which can be considered the sensitivity of the PGMY-LB assay compared to the HPV DNA reference standard, ranged from 95 to 100% for HPV DNA detection and typing using the consensus definition of HPV results and from 90 to 100% using results from the first test (Table 4). Samples containing multiple HPV types or with high copy numbers of HPV DNA were all appropriately classified by each site (data not shown).

In laboratories A and B, amplicons generated by PGMY primers were tested with the LB assay and also with the generic probe microplate assay. Since two laboratories tested 109 samples in triplicate, 654 assay results could be compared. Indeterminate results for the generic probe microplate assay were obtained for 59 samples (30 HPV-positive samples and 29 HPV-negative samples). When these indeterminate results were excluded, 420 assays were HPV positive by both tests, 149 were negative by both tests, 11 were HPV positive with the generic probe microplate assay only, and 15 were HPV-positive with the LB assay only. The agreement between the tests for detection of HPV amplicons was 95.6% (569 of 595 results), for an excellent kappa value of 0.89.


arrow
DISCUSSION
 
Interlaboratory agreement is a concern for any HPV DNA detection method that has not undergone extensive comparative field testing. The HPV detection and typing method also has to be accurate. We evaluated the intralaboratory and interlaboratory reproducibility of the PGMY-LB assay performed by three laboratories with different levels of experience with PCR testing. Our study demonstrates the high reliability of the PGMY-LB assay when a laboratory is well trained to perform the assay. Laboratories without experience with in-house PCR tests can also use the PGMY-LB assay, since laboratory C generated reproducible and accurate results. The use of standardized reagents and clearly written protocols was instrumental in obtaining these excellent results with the PGMY-LB assay. The lowest intralaboratory, interlaboratory, and accuracy levels were found in a laboratory with experience with PCR testing. That laboratory had determined in previous experiments that the PGMY-LB assay with their thermocycler required higher concentrations of ß-globin primers in the PCR master mix (data not shown). It was demonstrated previously that coamplification of ß-globin with HPV DNA can result in competition and impede PGMY-LB assay performance, explaining in part the discordant results obtained with low-copy-number synthetic samples (3, 4). The importance of utilizing the same instruments and parameters set by validation studies of a test is emphasized by these results. Each of the three laboratories used a different make and model of thermocycler, while the PGMY-LB assay protocol was developed and optimized for only one, the TC 9600 from Perkin-Elmer. As an alternative explanation, specimen heterogeneity may have accounted for the difference in performance between laboratories.

We demonstrated on synthetic samples that specimens with small amounts of HPV DNA were prone to generate discordant results between triplicates. Since synthetic samples do not contain the amplification modifier found in clinical specimens, we also included in the proficiency panel anonymous cytobrush samples. Ten clinical samples contained at least one HPV type not consistently detected by the reference laboratory. For all of these samples, the type detected intermittently by the reference laboratory was also detected by at least one of the participating laboratories. This phenomenon could result from the presence of a low copy number of HPV DNA or inhibitors and illustrates the difficulty of using clinical samples in a proficiency test.

The intralaboratory agreement was excellent for all three laboratories. It increased when we did not consider variable HPV results obtained in the certification panel by the reference laboratory. Some groups have reported greater reproducibility of PCR for HPV testing (22, 27). In those studies, duplicates instead of triplicates were tested, only one laboratory was evaluated, and the panel included more HPV-negative samples than did our study. Our level of intralaboratory agreement for HPV DNA detection and typing corresponds to a previous evaluation by Daniel et al. (6). They also reported that the reproducibility of HPV typing with MY09/MY11 increased in the presence of greater HPV loads. We found that for the laboratory with several discordant triplicates, low-viral-load synthetic samples were prone to generating discordant results. As reported by others, we found a lower agreement for HPV typing than for HPV DNA detection (6, 19). We calculated not only overall agreement but also agreement with regard only to HPV-positive results because of the large number of concordant HPV-negative assays that artificially increased the level of agreement for HPV typing. As previously reported, agreement was strong for samples with multiple HPV types, although we only analyzed three such specimens (6, 19).

Two studies reported the interlaboratory agreement of PCR for HPV detection using the MY09/MY11 primer pair (16, 22). In one study, 33 samples were tested in two laboratories, while in the other study, 70 anal samples were analyzed in two laboratories. Our evaluation of agreement was consistent with the estimates from these previous reports for HPV DNA positivity (83 and 96%) and HPV typing (90 and 88%).

The accuracy of the PGMY-LB assay was excellent. However, specimens may better agree with a reference standard when the same test is used to validate and test the panel. The low rate of false-positive results in the 29 HPV-negative samples suggests a low rate of contamination in our study. One of the false-positive specimens was preceded by an HPV-51-positive sample. Contamination may have occurred after amplification during the hybridization of amplicons with the LBs, since the same amplicons hybridized with the generic probe tested negative. Another study evaluated the accuracy of the GP5+/GP6+ L1 consensus primer pair and reported excellent accuracy for 50 samples tested in four laboratories, ranging from 92 to 100% for detection of HPV DNA, a rate similar to our results (19). That study also reported that most discrepant results were false-negative results from HPV-positive samples (19). The reproducibility of the PGMY-LB assay in our study was also similar to that of the FDA-approved hybrid capture assay from Digene (29). The accuracy of hybrid capture in detecting high-risk HPV types ranged for three laboratories from 88 to 92%, and interlaboratory agreement reached 90%.

Nonetheless, our results should be interpreted cautiously, with the following caveats. Few samples with multiple type infections were included in our evaluation. These samples have been demonstrated to generate intralaboratory variability in one study (6). Future evaluations of the PGMY-LB assay should include multiple type infections with some types in small amounts. Our clinical samples were obtained without knowledge of cytopathological diagnoses. Assay reproducibility and accuracy may differ when samples from different patient populations are selected, especially since viral load is related to cervical disease status and assay reproducibility (31).

We demonstrated here that there is a very strong interlaboratory agreement when the PGMY-LB assay is used. HPV testing can thus be reliably accomplished with PCR by using a validated and clearly written protocol with quality-controlled reagents. Although the assay procedure is well standardized, an initial certification panel to ensure the reliability of results would be desirable before a laboratory performs the PGMY-LB assay on a regular basis. Our study stresses the importance of using standardized reagents and protocols and the establishment of proficiency panels for the comparability of results between studies. More studies on performance standards that include samples with multiple type infections, with inhibitors and low viral load infections, need to be performed.


arrow
ACKNOWLEDGMENTS
 
We thank Diane Gaudreault and France Dion for testing samples with the PGMY-LB assay, Pierre Forest for training participants, and BMI Health Services for the provision of clinical samples. Roche Molecular Systems supplied reagents for the PGMY-LB assay.

F.C. and M.R. hold a clinical career award supported by the Fonds de Recherche en Santé du Québec.


arrow
FOOTNOTES
 
* Corresponding author. Mailing address: Département de Microbiologie et Infectiologie, Hôpital Notre-Dame du Centre Hospitalier de l'Université de Montréal, 1560 Sherbrooke est, Montréal, Québec H2L 4M1, Canada. Phone: (514) 890-8000, ext. 25162. Fax: (514) 412-7512. E-mail: francois.coutlee{at}ssss.gouv.qc.ca. Back


arrow
REFERENCES
 
    1
  1. Bauer, H. M., Y. Ting, C. E. Greer, J. C. Chambers, C. J. Tashiro, J. Chimera, A. Reingold, and M. M. Manos. 1991. Genital human papillomavirus infection in female university students as determined by a PCR-based method. JAMA 265:472-477.[Abstract/Free Full Text]
    2. 2
  2. Cope, J. U., A. Hildesheim, M. H. Schiffman, M. M. Manos, A. T. Lorincz, R. D. Burk, A. G. Glass, C. Greer, J. Buckland, K. Helgesen, D. R. Scott, M. E. Sherman, R. J. Kurman, and K. L. Liaw. 1997. Comparison of the Hybrid Capture tube test and PCR for detection of human papillomavirus DNA in cervical specimens. J. Clin. Microbiol. 35:2262-2265.[Abstract]
    3. 3
  3. Coutlée, F., P. Gravitt, J. Kornegay, C. Hankins, H. Richardson, N. Lapointe, H. Voyer, The Canadian Women's HIV study Group, and E. Franco. 2002. Use of PGMY primers in L1 consensus PCR improves detection of human papillomavirus DNA in genital samples. J. Clin. Microbiol. 40:902-907.[Abstract/Free Full Text]
    4. 4
  4. Coutlée, F., P. Gravitt, H. Richardson, C. Hankins, E. Franco, N. Lapointe, H. Voyer, and The Canadian Women's HIV Study Group. 1999. Nonisotopic detection and typing of human papillomavirus DNA in genital samples by the line blot assay. J. Clin. Microbiol. 37:1852-1857.[Abstract/Free Full Text]
    5. 5
  5. Coutlée, F., M. H. Mayrand, D. Provencher, and E. Franco. 1997. The future of HPV testing in clinical laboratories and applied virology research. Clin. Diagn. Virol. 8:123-141.[CrossRef][Medline]
    6. 6
  6. Daniel, R. W., L. Ahdieh, D. Hayden, S. Cu-Uvin, and K. V. Shah. 2000. Intra-laboratory reproducibility of human papillomavirus identification in cervical specimens by a polymerase chain reaction-based assay. J. Clin. Virol. 19:187-193.[CrossRef][Medline]
    7. 7
  7. de Roda, H., J. M. Walboomers, A. J. van den Brule, C. J. Meijer, and P. J. Snijders. 1995. The use of general primers GP5 and GP6 elongated at their 3' ends with adjacent highly conserved sequences improves human papillomavirus detection by PCR. J. Gen. Virol. 76:1057-1062.[Abstract/Free Full Text]
    8. 8
  8. Ferenczy, A., and E. L. Franco. 2002. Persistent human papillomavirus infection and cervical neoplasia. Lancet Oncol. 3:11-16.[CrossRef][Medline]
    9. 9
  9. Fleiss, J. L. 1981. Statistical methods for rates and proportions. John Wiley and Sons Inc., New York, N.Y.
    10. 10
  10. Franco, E. L. F. 1996. Epidemiology of anogenital warts and cancer, p. 597-623. In R. Reid and A. T. Lorincz (ed.), Human papillomavirus I. W. B. Saunders, Philadelphia, Pa.
    11. 11
  11. Gravitt, P., A. Hakenewerth, and J. Stoerker. 1991. A direct comparison of methods proposed for use in widespread screening of human papillomavirus infection. Mol. Cell. Probes 5:65-72.[CrossRef][Medline]
    12. 12
  12. Gravitt, P., C. L. Peyton, T. Q. Alessi, C. Wheeler, F. Coutlée, A. Hildesheim, M. Schiffman, D. R. Scott, and R. J. Apple. 2000. Improved amplification of genital human papillomaviruses. J. Clin. Microbiol. 38:357-361.[Abstract/Free Full Text]
    13. 13
  13. Gravitt, P., C. L. Peyton, R. J. Apple, and C. Wheeler. 1998. Genotyping of 27 human papillomavirus types by using L1 consensus PCR products by single-hybridization, reverse line blot detection method. J. Clin. Microbiol. 36:3020-3027.[Abstract/Free Full Text]
    14. 14
  14. Guerrero, E., R. W. Daniel, F. X. Bosch, X. Castellsagué, N. Munoz, M. Gili, P. Viladu, C. Navarro, M. L. Zubiri, N. Ascunce, L. C. Gonzales, L. Tafur, I. Izaraugaza, and K. V. Shah. 1992. Comparison of Virapap, Southern blot hybridization, and polymerase chain reaction methods for human papillomavirus identification in an epidemiological investigation of cervical cancer. J. Clin. Microbiol. 30:2951-2959.[Abstract/Free Full Text]
    15. 15
  15. Hildesheim, A., M. H. Schiffman, P. E. Gravitt, A. G. Glass, C. E. Greer, T. Zhang, D. R. Scott, B. B. Rush, P. Lawler, M. E. Sherman, R. J. Kurman, and M. M. Manos. 1994. Persistence of type-specific human papillomavirus infection among cytologically normal women. J. Infect. Dis. 169:235-240.[Medline]
    16. 16
  16. Hsing, A. W., R. D. Burk, K. L. Liaw, C. J. Chen, T. Zhang, M. Schiffman, C. E. Greer, S. L. You, C. Y. Hsieh, T. W. Huang, T. C. Wu, T. J. O'Leary, J. D. Seidman, and M. M. Manos. 1996. Interlaboratory agreement in a polymerase chain reaction-based human papillomavirus DNA assay. Cancer Epidemiol. Biomed. Prev. 5:483-484.[Medline]
    17. 17
  17. Jackson, J. B., J. Drew, H. J. Lin, P. Otto, J. W. Bremer, F. B. Hollinger, S. M. Wolinski, The ACTG PCR Working Group, and The ACTG PCR Virology Laboratories. 1993. Establishment of a quality assurance program for human immunodeficiency virus type 1 DNA polymerase chain reaction assays by the AIDS clinical trials group. J. Clin. Microbiol. 31:3123-3128.[Abstract/Free Full Text]
    18. 18
  18. Jacobs, M. V., P. J. F. Snijders, A. J. C. van den Brule, T. J. M. Helmerhorst, C. Meijer, and J. Walboomers. 1997. A general primer GP5+/GP6+-mediated PCR-enzyme immunoassay method for rapid detection of 14 high-risk and 6 low-risk human papillomavirus genotypes in cervical scrapings. J. Clin. Microbiol. 35:791-795.[Abstract]
    19. 19
  19. Jacobs, M. V., P. J. F. Snijders, F. J. Voorhorst, J. Dillner, O. Forslund, B. Johansson, M. V. K. Doeberitz, C. J. L. M. Meijer, T. Meyer, I. Nindl, H. Pfister, E. Stockfleth, A. Strand, G. Wadell, and J. M. M. Walboomers. 1999. Reliable high risk HPV DNA testing by polymerase chain reaction: an intermethod and intramethod comparison. J. Clin. Pathol. 52:790.
    20. 20
  20. Kleter, B., L. J. van Doorn, L. Schrauwen, A. Molijn, S. Sastrowijoto, J. ter Schegget, J. Lindeman, B. ter Harmsel, M. Burger, and W. Quint. 1999. Development and clinical evaluation of a highly sensitive PCR-reverse hybridization line probe assay for detection and identification of anogenital human papillomavirus. J. Clin. Microbiol. 37:2508-2517.[Abstract/Free Full Text]
    21. 21
  21. Kornegay, J. R., A. P. Shepard, C. Hankins, E. Franco, N. Lapointe, H. Richardson, The Canadian Women's HIV Study Group, and F. Coutlée. 2001. Nonisotopic detection of human papillomavirus DNA in clinical specimens using a consensus PCR and a generic probe mix in an enzyme-linked immunoassay format. J. Clin. Microbiol. 39:3530-3536.[Abstract/Free Full Text]
    22. 22
  22. Kuypers, J. M., C. W. Critchlow, P. E. Gravitt, D. A. Vernon, J. B. Sayer, M. M. Manos, and N. B. Kiviat. 1993. Comparison of dot filter hybridization, Southern transfer hybridization, and polymerase chain reaction amplification for diagnosis of anal human papillomavirus infection. J. Clin. Microbiol. 31:1003-1006.[Abstract/Free Full Text]
    23. 23
  23. Manos, M. M., Y. Ting, D. K. Wright, A. J. Lewis, T. Broker, and S. M. Wolinski. 1989. Use of the polymerase chain reaction amplification for the detection of genital human papillomaviruses. Cancer Cells 7:209-215.
    24. 24
  24. Nobbenhuis, M. A., J. M. Walboomers, T. J. Helmerhorst, L. Rozendaal, A. J. Remmink, E. K. Risse, H. C. van der Linden, F. J. Voorhorst, P. Kenemans, and C. J. Meijer. 1999. Relation of human papillomavirus status to cervical lesions and consequences for cervical-cancer screening: a prospective study. Lancet 354:20-25.[CrossRef][Medline]
    25. 25
  25. Noordhoek, G. T., A. H. Kolk, G. Bjune, D. Catty, J. W. Dale, P. E. Fine, P. Godfrey-Faussett, S. N. Cho, T. Shinnick, and S. B. Svenson. 1994. Sensitivity and specificity of PCR for detection of Mycobacterium tuberculosis: a blind comparison study among seven laboratories. J. Clin. Microbiol. 32:277-284.[Abstract/Free Full Text]
    26. 26
  26. Peyton, C. L., M. Schiffman, A. T. Lorincz, W. C. Hunt, I. Mielzynska, C. Bratti, S. Eaton, A. Hildesheim, L. A. Morera, A. C. Rodriguez, R. Herrero, M. E. Sherman, and C. M. Wheeler. 1998. Comparison of PCR- and hybrid capture-based human papillomavirus detection systems using multiple cervical specimen collection strategies. J. Clin. Microbiol. 36:3248-3254.[Abstract/Free Full Text]
    27. 27
  27. Qu, W., G. Jiang, Y. Cruz, C. J. Chang, G. Y. Ho, R. S. Klein, and R. D. Burk. 1997. PCR detection of human papillomavirus: comparison between MY09/MY11 and GP5+/GP6+ primer systems. J. Clin. Microbiol. 35:1304-1310.[Abstract]
    28. 28
  28. Schiffman, M. H., H. M. Bauer, and A. T. Lorincz. 1991. Comparison of Southern blot hybridization and polymerase chain reaction methods for the detection of human papillomavirus DNA. J. Clin. Microbiol. 29:573-577.[Abstract/Free Full Text]
    29. 29
  29. Schiffman, M. H., N. B. Kiviat, R. D. Burk, K. V. Shah, R. W. Daniel, R. Lewis, J. Kuypers, M. M. Manos, D. R. Scott, M. E. Sherman, R. J. Kurman, M. H. Stoler, A. G. Glass, B. B. Rush, I. Mielzynska, and A. T. Lorincz. 1995. Accuracy and interlaboratory reliability of human papillomavirus DNA testing by hybrid capture. J. Clin. Microbiol. 33:545-550.[Abstract]
    30. 30
  30. Smits, H. L., L. J. M. Bollen, S. P. Tjong-A-Hung, J. Vonk, J. van der Vendel, F. J. W. Ten Kate, J. A. Kaan, B. W. Mol, and J. ter Schegget. 1995. Intermethod variation in detection of human papillomavirus DNA in cervical smears. J. Clin. Microbiol. 33:2631-2636.[Abstract]
    31. 31
  31. Swan, D. C., R. A. Tucker, G. Tortolero-Luna, M. F. Mitchell, L. Wideroff, E. R. Unger, R. A. Nisenbaum, W. C. Reeves, and J. P. Icenogle. 1999. Human papillomavirus (HPV) DNA copy number is dependent on grade of cervical disease and HPV type. J. Clin. Microbiol. 37:1030-1034.[Abstract/Free Full Text]
    32. 32
  32. Tang, Y. W., and D. H. Persing. 1999. Molecular detection and identification of microorganisms, p. 215-244. In P. R. Murray, E. J. Baron, M. A. Pfaller, F. C. Tenover, and R. H. Yolken (ed.), Manual of clinical microbiology, 7th ed. American Society for Microbiology, Washington, D.C.
    33. 33
  33. Vernon, S. D., E. R. Unger, and D. Williams. 2000. Comparison of human papillomavirus detection and typing by cycle sequencing, line blotting, and Hybrid Capture. J. Clin. Microbiol. 38:651-655.[Abstract/Free Full Text]
    34. 34
  34. Villa, L. L. 1997. Human papillomaviruses and cervical cancer. Adv. Cancer Res. 71:321-341.[Medline]

Journal of Clinical Microbiology, March 2003, p. 1080-1086, Vol. 41, No. 3
0095-1137/03/$08.00+0     DOI: 10.1128/JCM.41.3.1080-1086.2003
Copyright © 2003, American Society for Microbiology. All Rights Reserved.



This article has been cited by other articles:

  • Wolffs, P. F. G., Vink, C., Keijdener, J., Habek, B., Reijans, M., Simons, G., Bruggeman, C. A., van den Brule, A. J. C. (2009). Evaluation of MeningoFinder, a Novel Multiplex Ligation-Dependent Probe Amplification Assay for Simultaneous Detection of Six Virus Species Causing Central Nervous System Infections. J. Clin. Microbiol. 47: 2620-2622 [Abstract] [Full Text]  
  • Koshiol, J., Lindsay, L., Pimenta, J. M., Poole, C., Jenkins, D., Smith, J. S. (2008). Persistent Human Papillomavirus Infection and Cervical Neoplasia: A Systematic Review and Meta-Analysis. Am J Epidemiol 168: 123-137 [Abstract] [Full Text]  
  • van Hamont, D., van Ham, M. A. P. C., Bakkers, J. M. J. E., Massuger, L. F. A. G., Melchers, W. J. G. (2006). Evaluation of the SPF10-INNO LiPA Human Papillomavirus (HPV) Genotyping Test and the Roche Linear Array HPV Genotyping Test.. J. Clin. Microbiol. 44: 3122-3129 [Abstract] [Full Text]  
  • Coutlee, F., Rouleau, D., Petignat, P., Ghattas, G., Kornegay, J. R., Schlag, P., Boyle, S., Hankins, C., Vezina, S., Cote, P., Macleod, J., Voyer, H., Forest, P., Walmsley, S., The Canadian Women's HIV Study Group, , Franco, E. (2006). Enhanced Detection and Typing of Human Papillomavirus (HPV) DNA in Anogenital Samples with PGMY Primers and the Linear Array HPV Genotyping Test.. J. Clin. Microbiol. 44: 1998-2006 [Abstract] [Full Text]  
  • Suba, E. J., Murphy, S. K., Donnelly, A. D., Furia, L. M., Huynh, M. L. D., Raab, S. S. (2006). Systems Analysis of Real-World Obstacles to Successful Cervical Cancer Prevention in Developing Countries. AJPH 96: 480-487 [Abstract] [Full Text]  
  • Quint, W. G. V., Pagliusi, S. R., Lelie, N., de Villiers, E.-M., Wheeler, C. M., the World Health Organization Human Papillomavirus, (2006). Results of the First World Health Organization International Collaborative Study of Detection of Human Papillomavirus DNA. J. Clin. Microbiol. 44: 571-579 [Abstract] [Full Text]  

This Article
Right arrow Abstract Freely available
Right arrow Full Text (PDF)
Right arrow Alert me when this article is cited
Right arrow Alert me if a correction is posted
Services
Right arrow Similar articles in this journal
Right arrow Similar articles in PubMed
Right arrow Alert me to new issues of the journal
Right arrow Download to citation manager
Right arrowReprints and Permissions
Right arrow Copyright Information
Right arrow Books from ASM Press
Right arrow MicrobeWorld
Citing Articles
Right arrow Citing Articles via HighWire
Right arrow Citing Articles via Google Scholar
Google Scholar
Right arrow Articles by Kornegay, J. R.
Right arrow Articles by Coutlée, F.
Right arrow Search for Related Content
PubMed
Right arrow PubMed Citation
Right arrow Articles by Kornegay, J. R.
Right arrow Articles by Coutlée, F.

Copyright © 2009 by the American Society for Microbiology.   For an alternate route to Journals.ASM.org, visit: http://intl-journals.asm.org | More Info»