INTRODUCTION

Top Abstract Introduction Materials and Methods Results Discussion References

Direct detection and quantitation of hepatitis B virus (HBV) DNA in plasma or serum are now used routinely to evaluate viremia in HBV-infected persons, to identify infectious chronic carriers, and to predict and monitor the efficacy of antiviral therapy (2, 8, 11). Since the early 1980s, a variety of molecular detection and quantitation methods have been developed, including dot and slot blot hybridization with radioactive and nonradioactive DNA probes (19-21), chemiluminescent detection of HBV DNA-RNA hybrids (14), PCR amplification of HBV DNA followed by hybridization to probes bound to a microwell plate (10, 12, 22) or magnetic beads (13), branched DNA (bDNA) signal amplification of an HBV DNA-DNA hybrid (7), transcription-mediated amplification (9), and fluorescent real-time detection of amplified HBV DNA (1). Each method, calibrated uniquely, exhibits its own sensitivity, specificity, and dynamic range. Standardization is ongoing (5, 6).

To assess the relative value of these methods in detecting and quantitating HBV DNA, international proficiency studies with well-characterized, simulated clinical samples would be required. In the first and only such study published to date (17), 39 laboratories analyzed 22 samples, including 12 undiluted samples with and without HBV DNA. (The lowest positive sample contained 3.5 pg/ml, or approximately 980,000 copies/ml.) Only 27.9% of the data sets had all 12 samples correct, and 34.9% showed false-positive results. Clearly, a majority of the participating laboratories had problems with both sensitivity and specificity.

The present report describes two recent HBV proficiency panels (lowest viral load of 1,000 copies/ml) designed by the European Union Concerted Action on Quality Control (EU QCCA) of Nucleic Acid Amplification in Diagnostic Virology and prepared by Boston Biomedica, Inc. (BBI; West Bridgewater, Mass.). The results obtained with these panels demonstrate that while the qualitative detection of HBV DNA has significantly improved, the detection levels of many commercial quantitative assays are still too high to allow adequate quantitation of the clinical samples seen in routine diagnostic laboratories.

	MATERIALS AND METHODS

Top Abstract Introduction Materials and Methods Results Discussion References

Panels. (i) Preparation. Panels were prepared by BBI from human plasma containing HBV DNA of subtype ad or ay by appropriate dilution in sterile filtered defibrinated plasma (Basematrix) with 0.09% sodium azide as preservative in accordance with the ISO 9001 Quality System Standards and the 21CFR 820 "Good Manufacturing Practice for Medical Devices: General." Plasma units were obtained from Food and Drug Administration-licensed facilities that comply with the applicable federal regulations (21CFR, part 600).

(ii) Composition. Each panel consisted of eight coded samples. Six samples contained HBV DNA with approximate target levels of 10³ to 10⁷ copies/ml (panel 1) and 10³ to 2 × 10⁶ copies/ml (panel 2). Two samples contained no virus and served as negative controls. To evaluate interassay reproducibility, three identical samples were included in both panels: 2 × 10⁶ copies/ml each for ad and ay and 2 × 10⁵ copies/ml for ad. To assess a possible effect of HBV subtype, each panel contained two pairs of samples with identical viral loads, but different subtypes.

Participants. The panels were distributed on dry ice by courier service to 45 laboratories (panel 1) and 61 laboratories (panel 2). The recipient laboratories were asked to report the arrival and condition of the panel immediately by fax and to return the results as soon as possible, but within 6 weeks (panel 1) or 4 weeks (panel 2). A code number, known only to the Neutral Office, University of Manchester, Manchester, United Kingdom, identified each laboratory. Laboratories participating in both proficiency studies were assigned the same code for both panels. A questionnaire was also sent to obtain technical information on the procedures employed. To ensure confidentiality, all laboratories sent their results to the Neutral Office. The results were analyzed anonymously at the Department of Molecular Biology, Laboratory Dr. Schiwara and Partners, Bremen, Germany.

Qualitative analysis. For qualitative analysis, the results from the quantitative data sets were converted to qualitative data (i.e., positive or negative) and considered together with the true qualitative data sets. To assess performance, the following scoring system was applied: 1 point was given for each correct result for the true-positive and true-negative samples. A point was deducted for each false-positive or false-negative result, with the exception of the weak-positive sample containing 10³ copies/ml. Thus, the maximum score to be obtained was 8 points, which was qualified as "good." Scores of 7 and 6 points were considered "adequate" and "mediocre," respectively, while <6 points was considered "poor."

Quantitative analysis. For quantitative analysis, the overall geometric mean (GM) and standard deviations (SD) were calculated for each (positive) sample for all assays, as well as separately according to the method used. To assess performance, the following scoring system was utilized: 1 point was awarded for each viral load result that was within the range of ±0.5 log₁₀ of the overall GM of each sample. Scoring was for positive samples only. For all other results, i.e., each positive result outside this range, false-positive results, or negative results on positive samples, no point was given or deducted. The maximum score of 6 points was considered "good," 5 and 4 points were considered "adequate" and "mediocre," respectively, and <4 points was considered "poor."

	RESULTS

Top Abstract Introduction Materials and Methods Results Discussion References

Participants and methods. In panel 1, 42 laboratories from 19 countries submitted 20 qualitative (all in-house PCRs) and 37 quantitative (87% commercial assays) data sets. In panel 2, 51 laboratories from 18 countries submitted 25 qualitative (all in-house PCRs) and 47 quantitative (94% commercial assays) data sets. The methods utilized and the corresponding detection limits as reported by the participants are listed in Table 1.

Qualitative analysis. (i) Panel 1. Correct results for the two negative samples were reported in 52 of the 57 data sets (91.2%), and a false-positive result was obtained in 5 data sets, 1 of which contained two false-positive results (Table 2). Of these six false-positive results (6 of 114 negative samples = 5.3%), three were obtained with commercial kits (two by Digene Hybrid Capture Systems and one by Roche Monitor Assay). The weak-positive sample (10³ copies/ml) was correctly reported positive in 30 data sets. The 27 data sets with false-negative results were obtained with the Digene Hybrid Capture Systems (13 of 13), an in-house PCR (7 of 15), an in-house nested PCR (4 of 10), the Roche Monitor Assay (1 of 16), the Bayer bDNA Assay (1 of 2), and the BAG AcuGen Test (1 of 1). Another low-positive sample (2 × 10⁵ copies/ml) was correctly reported positive in 48 data sets. The nine false-negative results with this sample were in data sets obtained with the Digene Hybrid Capture Systems (7 of 13) and an in-house PCR (2 of 15). All other positive samples (2 × 10⁶ copies/ml) were correctly identified as positive in all data sets. The overall false-negative result rate was therefore 10.5% [36/(6 × 57] (Table 2).

(ii) Panel 2. Correct results for the two negative samples were reported in 70 data sets. A false-positive result was obtained in two qualitative data sets (both in-house PCRs); none of the quantitative data sets contained false-positive results (Table 2). Thus, the false-positive result rate was 1.4% (2 of 144 negative samples).

Quantitative analysis. (i) Panel 1. Quantitative HBV data were reported in 37 data sets, whereby results from 2 data sets (1 in-house PCR, all results listed as <8 × 10³;1 in-house nested PCR, all positive results listed as >10⁶) could not be included in the calculations. Most data sets (87%) were derived from commercial kits. The overall GM and SD were calculated for each (positive) sample from all assays (Table 4) as well as separately for the 16 Roche data sets, the 13 Digene data sets, and the 6 remaining data sets taken together (data not shown). The overall GM compared well with the target levels. For most samples, 89% of the positive results were within the range of GM ± 0.5 log₁₀. The GM values obtained with the Roche and Digene assays were remarkably similar and likewise corresponded well to the target levels (except for the weak-positive sample, which could not be detected by the Digene assays), although the SD values for the Roche assay were consistently larger than those for the Digene assays (data not shown). The GM values for the six remaining data sets calculated together were consistently lower than the GM for the Roche and Digene assays and showed consistently larger SD values (data not shown).

(ii) Panel 2. Quantitative HBV data were reported in 47 data sets, most (94%) derived from commercial kits. The overall GM and SD were calculated for each (positive) sample from all assays (Table 4), as well as separately according to the method used (data not shown). The overall GM corresponded well with the target levels. For most samples, 83% of the positive results were within the range of GM ± 0.5 log₁₀. As in panel 1, the GM for the Roche and Digene assays were similar but with larger SD values for the Roche assays (data not shown). The GM for the remaining assays tended to be lower than for the Roche and Digene assays, with the largest SD values obtained with the in-house quantitative PCRs (data not shown).

Reproducibility. Interpanel reproducibility could be evaluated from the results obtained with three samples represented in both panels (Table 6). For the ay subtype sample with 2 × 10⁶ copies/ml, the qualitative detection rates were virtually the same: 100 and 99%, respectively. However, the percentage of positive results within the range of ±0.5 log₁₀ of the GM for that sample was higher in panel 1 (91%) than in panel 2 (85%). The results for the ad subtype sample with the same viral load showed the same pattern: nearly identical qualitative detection rates (100 and 99%, respectively), but a higher percentage of positive results within the range of ±0.5 log₁₀ of the GM in panel 1 (89%) compared to that in panel 2 (83%).

	DISCUSSION

Top Abstract Introduction Materials and Methods Results Discussion References

To assess the value of currently available methods as utilized by diagnostic laboratories for detecting and quantitating HBV DNA, international proficiency studies with well-characterized, simulated clinical samples are required. In the first such study performed in conjunction with the European Expert Group on Viral Hepatitis (17), 39 laboratories submitted 43 data sets for 22 samples, including 12 undiluted samples (7 positive, 5 negative) with viral loads from 3.5 pg/ml (approximately 980,000 copies/ml) to 222 pg/ml (approximately 62,160,000 copies/ml). Viral loads were determined by a single assay (Genostics liquid hybridization test from Abbott). All but one laboratory used the PCR technique; none used the uracil N-glycosylase system to prevent contamination from previously amplified products. Of the 43 data sets, 15 (35%) showed false-positive results for the five negative samples. Furthermore, despite the relatively high value of the weakest sample (980,000 copies/ml), 16 of 43 data sets (37%) showed false-negative results. Only 12 of 43 data sets (28%) had all 12 samples correct. No quantitative analysis of the data was performed. The first study, however, clearly demonstrated the large number of laboratories with sensitivity and specificity problems in detecting HBV DNA.

Similar problems with sensitivity and specificity were reported from the early proficiency studies for Mycobacterium tuberculosis (16) and for hepatitis C virus (HCV) (24) (29% false-positive results and 39% false-negative results on 10 undiluted samples4 positive and 6 negativein 31 data sets). In the second HCV proficiency study, performed 3 years later (4), little improvement was reported (21% false-positive results, but 63% false-negative results for 10 undiluted samplesagain 4 positive and 6 negativein 136 data sets).

Compared to the first HBV proficiency study (17), the present study shows considerable improvement both in panel conception and results obtained. First, the samples in both panels were well characterized, having been defined by the manufacturers of several quantitative methods (Roche, Bayer, and Digene) and by three reference laboratories utilizing various assays. The close approximation of the GM of all test results from the participating laboratories supports the accuracy of the assigned target viral load. Secondly, the sample with the lowest viral load in both panels contained only 1,000 copies of HBV DNA per ml, reflecting more accurately the viral loads from patients undergoing antiviral therapy. Third, this study involved a large number of participants (42 and 51 in panels 1 and 2, respectively) and data sets (57 and 72 in panels 1 and 2, respectively). In addition, the participating laboratories are now utilizing a variety of methods for detecting and quantitating HBV DNA in addition to PCR, allowing such methods to be assessed in an international proficiency panel. Fourth, the data were analyzed not only qualitatively, but also quantitatively, by using a simple algorithm permitting comparison of the seven different quantitative assays employed. Finally, the description of two independent proficiency panels permits interpanel reproducibility testing.

One of the most significant results derived from this study is the lowest false-positive rate reported to date for any large proficiency panel. Similar low false-positive rates have been found in the EU QCCA HCV RNA and human immunodeficiency virus proficiency studies carried out simultaneously with this study (J. Schirm, A. M. van Loon, E. Valentine-Thon, J. Reid, P. E. Klapper, and G. M. Cleator, submitted for publication; A. M. van Loon, J. Schirm, E. Valentine-Thon, J. Reid, P. E. Klapper, and G. M. Cleator, unpublished data) and in the EU QCCA enterovirus proficiency study carried out several months earlier (23). Indeed, the rate of 8.8% (5 of 57) false-positive data sets or 5.3% (6 of 114) false-positive results for panel 1 improved even further to 2.8% (2 of 72) false-positive data sets or 1.4% (2 of 144) false-positive results for panel 2. These low rates may simply reflect the greater expertise of the participating laboratories compared to several years ago. The improvement may also result from the increasing use of commercial kits, many of which contain contamination control enzyme systems. Indeed, 56% of data sets in panel 1 and 61% of data sets in panel 2 were obtained from commercial kits. On the other hand, the use of commercial kits does not safeguard against false-positive results: while the only false-positive results in panel 2 were obtained with in-house PCRs, 3 of 6 false-positive results in panel 1 were obtained with commercial kits. Finally, the lower false-positive rate in panel 2 compared to that in panel 1 may also represent a beneficial effect of participation in this proficiency program: of the four laboratories generating false-positive results in panel 1, three participated in panel 2 without producing false-positive results. Furthermore, the two laboratories generating false-positive results in panel 2 had not participated in panel 1.

In contrast to the low rate of false-positives in this study, the rate of negative results for (true) positive samples was remarkably high: for panel 1, 43.9% (25 of 57) of data sets, or 10.5% [36/(57 × 6] of results; for panel 2, 45.8% (33 of 72) of data sets, or 17.4% [75/(72 × 6] of results. Obviously, this high rate reflects the deliberate low viremic composition of the samples, in particular in panel 2 (2 × 10⁶ copies/ml) coupled with the high detection limits of some commercial kits still in use (i.e., Hybrid Capture System I with 1.4 × 10⁶ copies/ml or Bayer bDNA with 0.7 × 10⁶ copies/ml). (While the negative results obtained with low viremic samples by using such kits could arguably be excluded from the category "false-negative," we have chosen to include them in this category for the sake of simplicity.) As a result, for both qualitative and quantitative analyses, the percentage of data sets achieving scores of adequate or better decreased in panel 2 compared to that in panel 1. While it is recognized that such a panel composition "penalizes" users of commercial kits with high detection limits, the increasing clinical requirement for low-level dynamic ranges dictates this necessity. A potentially relevant assay is the newly introduced Cobas Amplicor HBV Monitor Test from Roche Diagnostics. However, while its low detection limit (200 copies/ml) is suitable for monitoring patients undergoing therapy, its upper dynamic range of only 200,000 copies/ml requires pretest dilution of HBsAg- and HBeAg-positive samples, concomitantly increasing costs (13, 15). Alternatively, transcription-mediated amplification followed by hybridization of two probes with different specific activities allows a broad detection range of 5 × 10³ to 5 × 10⁸ copies/ml (9). In contrast to Cobas Amplicor, however, this method contains no internal control for inhibition and is not yet automated.

In addition to low detection levels and a broad dynamic range, appropriate quantitation assays should be calibrated to internationally defined reference standards, as now available for HCV RNA (18). In the present study, the quantitative analysis showed variations ranging from 6 to 35% (percent positive results outside the defined range) in the actual copy numbers assigned to samples. Similar method-related deviations in quantitation have been reported by others (3, 5) In 1999, the Eurohep Pathobiology Group established two international reference plasma preparations, each containing approximately 2.6 × 10⁹ copies/ml, thereafter defined as 10⁹ Eurohep units (6). These Eurohep samples have been used for the evaluation of commercial kits (10, 14), and one of these may be the basis of a World Health Organization reference sample. In the present study, a panel 1 sample (ay subtype) with 10⁷ copies/ml could also serve as a candidate reference standard, because the GM ± SD log₁₀ for all 35 assays was 7.05, and 94% of the positive results with this sample were within the range of GM ± 0.5 log₁₀.

Finally, both qualitative and quantitative assays must yield reproducible results. The interpanel reproducibility of the three samples represented in both panels was excellent: overall qualitative detection rates were nearly identical. However, for the two samples with 2 × 10⁶ copies/ml, the percentage of positive results within the range of ±0.5 log₁₀ of the GM for that sample was higher in panel 1 than in panel 2, while for the lower viremic sample (2 × 10⁵ copies/ml), the opposite was true. Apparently, the increased use of commercial kits in panel 2 led to an increase in "within-range" detection rates in the lower dynamic rangeat the expense, however, of a decreased within-range detection rate in the upper dynamic range.

In conclusion, this extensive proficiency study demonstrates considerable improvement in the qualitative performance of currently available HBV DNA assays. At the same time, however, the results emphasize the need for commercial viral load kits with sufficiently low detection levels to permit adequate HBV DNA detection and quantitation in clinical samples.