TEXT

Top Abstract Text References

The quantification in plasma of human immunodeficiency virus type 1 (HIV-1) RNA is a useful test for predicting clinical outcome, evaluating disease progression, and monitoring the effect of antiretroviral treatment on HIV-1-infected patients (6, 9). The primary reason for undertaking this project was to evaluate the performance of selected methodologies or laboratories for monitoring the effect of antiretroviral therapy on HIV-1 viral load in clinical research. Three manufacturers of HIV-1 RNA kits and an independent laboratory agreed to participate in a single round of proficiency testing on a panel of plasma specimens. The laboratories performing the tests were chosen by the manufacturers. Enhanced-sensitivity bDNA assays (ES-bDNA) were performed at the Chiron Reference Testing Laboratory (Emeryville, Calif.), Roche AMPLICOR HIV-1 Monitor tests were carried out at Novum, Inc. (Pittsburgh, Pa.), and NASBA (nucleic acid sequence-based amplification) HIV-1 QT assays were performed at Advanced Bioscience Laboratories, Inc. (Kensington, Md.). National Genetics Institute, or NGI (Culver City, Calif.), an established independent laboratory, used its in-house method. Because the panel samples were prepared at the Center for AIDS Research Virology Core Facility at Baylor College of Medicine (BCM), we have also included in this report the results obtained with the in-house method that was used to establish their nominal concentrations.

Summarized in Table 1 are the differences among the five assays. The ES-bDNA assay for HIV-1 RNA is a second-generation assay developed by the Chiron Corporation (Emeryville, Calif.) and incorporates changes in probe design and reagent composition (3). The distinctive feature of the NGI in-house assay was the use of four different numbers of PCR cycles (1). The AMPLICOR HIV-1 Monitor kit, manufactured by Roche Molecular Systems (Branchburg, N.J.), employed Thermus thermophilus polymerase, a recombinant enzyme capable of carrying out both reverse transcription (RT) and PCR (7). Assays based on NASBA technology developed by Organon Teknika (Durham, N.C.) were carried out under isothermal conditions and resulted in an RNA amplification product (11). The BCM method employed the Qiagen viral RNA kit (Santa Clarita, Calif.) for sample preparation and chemiluminescence assay of PCR products (Digene Diagnostics, Inc., Silver Spring, Md.) (5).

HIV-1 standards and panel samples were prepared with normal acid-citrate-dextrose (ACD) plasma from individual blood donors who tested negative for anti-HIV-1, anti-HIV-2, anti-hepatitis B core, hepatitis B surface antigen, anti-hepatitis C virus, and anti-human T-cell leukemia virus type 1 (The Gulf Coast Regional Blood Center, Houston, Tex.). The primary standard was prepared by spiking negative plasma with supernatant fluid from a coculture of peripheral blood mononuclear cells from an HIV-1-infected patient (4). The secondary standard and panel samples were prepared by spiking negative plasma with HIV-1 strain HXB3 (5). Calibration of the secondary standards against the primary standard was based on 10 observations performed in five separate experiments with a between-runs coefficient of variation (CV) of 11.8%. The panel consisted of duplicate samples at 2.0 × 10⁶, 1.0 × 10⁶, 500,000, 130,000, and 65 HIV-1 RNA copies per ml; triplicate samples at 65,000, 13,000, 6,500, 1,300, 650, and 130 HIV-1 RNA copies per ml; and four samples of negative plasma. Aliquoting of samples was performed under optimal conditions to ensure sample uniformity. Specimens were shipped to testing sites on dry ice, and testing was performed in a blinded fashion at all sites, including the BCM laboratory.

Presented in Table 2 are the results of HIV-1 RNA quantification for each sample as reported by the testing sites, compared with the nominal HIV-1 RNA concentrations. In comparisons of those results that fell within the dynamic range claimed for each method (listed in a footnote to the table), the differences between the means of results for replicate samples that were obtained with the Chiron, NGI, BCM, and Roche assays generally fell within the same order of magnitude.

The data in Table 2 were evaluated with respect to the ability of the assays to detect the known differences in HIV-1 RNA concentration present in the dilution series (Fig. 1). Nominal and estimated HIV-1 RNA concentrations were expressed as logarithms to the base 10. For each method, the mean of the results for each group of two or three panel samples with the same nominal HIV-1 concentration was calculated and represented as a point in the appropriate panel in Fig. 1. Groups that contained results falling outside the claimed dynamic range were omitted. For example, the ES-bDNA assay results for the three samples with 500 nominal HIV-1 RNA copies per ml and two samples with nominal copy numbers of 2 × 10⁶ were omitted. If an assay method correctly estimated the relationship of the various samples in the dilution series, a log-log plot of the estimated mean HIV-1 RNA concentrations against the nominal concentrations should produce a straight line having the same slope as the line of equivalence, shown in each panel as a diagonal straight line. The data set exhibiting the closest approximation of a straight line parallel to the line of equivalence was obtained by ES-bDNA. With perfect accuracy, the line representing the assay results should fall on the line of equivalence. This criterion must be applied with discretion, since the nominal concentrations of the panel samples are at best only estimates of an unknown value, the true HIV-1 RNA concentration. The results obtained with the Chiron, NGI, BCM, and Roche assays fell closest to the line of equivalence. These four assays provided estimates of HIV-1 RNA concentrations that generally agreed with each other and with the nominal concentrations of the BCM standards. In this sense, these assays were accurate. The slope of the line for the NASBA test also paralleled the line of equivalence but was displaced from it by approximately one log₁₀.

View larger version (28K): [in this window] [in a new window]   FIG. 1.   Linearity of HIV-1 RNA estimates performed on serially diluted spiked specimens by different assay methods. Assays employed were ES-bDNA, NGI, BCM, AMPLICOR HIV-1 Monitor, and NASBA HIV-1 QT. The straight diagonal line represents the equivalence of nominal and estimated HIV-1 RNA concentrations. Estimated concentrations for the dilution series that were obtained for a given assay are shown as connected points. Each point represents the mean of two or three estimates. If one or more of the reported results for a point fell outside the dynamic range of the assay, that point was omitted.

The performance characteristics of the five assays are summarized in Table 3. The calculations of correlation, CV, and standard deviation (SD) were based solely on those results that fell within the claimed dynamic ranges. Correlation between the observed and nominal concentrations was close to 1.000 for all five assays, although the results obtained by the NASBA assay were consistently higher. Intra-assay precision was expressed by the CV and as the logarithm of the SD to the base 10 (log₁₀ SD), a calculation that conveniently allowed comparison of SDs when the range of values covered several orders of magnitude. The ES-bDNA assay showed the smallest mean CV. An intra-assay SD of 0.15 to 0.20 log₁₀ RNA copies per ml is required in order to achieve the degree of precision that will provide 90% power to detect a fivefold difference in RNA concentration between two samples (12). The ES-bDNA, NGI, and BCM assays achieved this degree of precision. The duplicate log₁₀ SD was lower than the triplicate log₁₀ SD for all methods. This result was attributable to the composition of the panel. Samples tested in duplicate were preponderantly those with high HIV-1 RNA concentrations, while samples tested in triplicate were those with lower concentrations. Of the five assays, the ES-bDNA assay showed the highest overall intra-assay precision, with a log₁₀ SD of 0.050 to 0.054. 

HIV-1-negative samples were correctly identified in all five assays, and no false positives were found; thus, the specificity was 100% for all five methods. It was demonstrated in the experiment shown in Fig. 1 that there was little or no difference between the nominal HIV-1 RNA concentrations and the estimates of HIV-1 RNA concentration obtained by the Chiron, NGI, BCM, and Roche methods. It was thus possible to compare the dynamic range claimed for each of these four methods to the observed lower and upper limits of detection, abbreviated L and U, respectively. The observed L for ES-bDNA was higher than the claimed L; the observed H was consistent with the claimed H. NGI's observed L and U exceeded its claimed dynamic range, while BCM's observed L and U were consistent with its claims. The observed L and U of the AMPLICOR HIV-1 Monitor were both lower than those claimed by the manufacturer.

This study underscores the importance of the use of well-defined standards for assigning nominal HIV-1 RNA concentrations to panel materials. In a previous comparison of the AMPLICOR HIV-1 Monitor and NASBA assays, both methods produced estimates below the nominal copy numbers (10). Compared to Monitor, NASBA has yielded higher estimates of HIV-1 RNA concentrations in two of three studies (2, 8, 10).

This article describes the first performance comparison of ES-bDNA with other assays for HIV-1 RNA. Among the five laboratories that used ES-bDNA, the NGI assay, the BCM assay, Roche HIV-1 Monitor, and NASBA HIV-1 QT, the laboratories performing the first four tests named provided results that were generally consistent with each other. However, interlaboratory variation with the same kit is known to occur and may be greater than that which occurs with different kits (12). In other evaluations, the NASBA assay has performed well (2, 8, 10). The reason for the approximately 10-fold-higher values provided by the NASBA assay in the present study is being investigated. This event appears to be isolated, since many laboratories have successfully adopted the NASBA method (12). In our opinion, the laboratory may be more important than the assay method in the selection of a testing site for clinical trials.