British Columbia Centers of Disease Control, Vancouver, British Columbia, Canada,1 Bayer Diagnostics, Berkeley,2 Bayer Reference Testing Laboratory, Emeryville,3 Clinical Research Consultant, Palo Alto, California4
Received 26 June 2001/ Returned for modification 15 December 2001/ Accepted 12 May 2002
| ABSTRACT |
|---|
|
|
|---|
| INTRODUCTION |
|---|
|
|
|---|
The qualitative Roche Cobas Amplicor HCV version 2.0 assay (HCV PCR), a reverse transcription-PCR assay, has a reported limit of detection of 50 IU/ml (11) and has been used to confirm viremia and to measure treatment response (10). Recent studies reported that another nucleic acid test based on transcription-mediated amplification (TMA), the HCV RNA transcription-mediated amplification qualitative test (HCV TMA), was able to detect HCV RNA in some patients who had no detectable HCV RNA (as determined by the first and second versions of HCV PCR) at the end of treatment and subsequently experienced virologic relapse (4, 18). These results suggested that HCV TMA is more sensitive than HCV PCR and that test sensitivity may be important for accurate assessment of antiviral response especially when measuring end-of-treatment viral clearance (6, 12, 23). We therefore assessed the limits of detection of the two assays with a dilution panel of the World Health Organization (WHO) international HCV standard, analyzed the interassay concordance with clinical specimens, and assessed assay throughput and workflow.
| MATERIALS AND METHODS |
|---|
|
|
|---|
HCV RNA TMA qualitative test (HCV TMA). Following a lysis step, target HCV RNA is captured to magnetic particles coated with oligonucleotides complementary to the 5' untranslated region of the HCV genome. Monitoring of target capture and amplification is achieved by adding an internal control RNA to each sample. The target RNA is then amplified with an isothermal TMA process that requires the addition of primers, reverse transcriptase, and T7 RNA polymerase. Detection of the amplified product is based on hybrid protection and the dual kinetic assays (18, 19). Each tube produces a chemiluminescent signal that is read as relative light units (RLU). Data are reported both as calculated RLU and as signal-to-cutoff ratios. When the signal-to-cutoff ratio is >1, the specimen is considered reactive or as having detectable HCV RNA. There are two possible outcomes of this assay; HCV RNA is either detected or not detected. Maximum throughput of HCV TMA in one run is 91 reportable specimens. A total of nine calibrators and controls are required.
WHO HCV dilution panel. A dilution panel of the WHO standard for HCV RNA (WHO 96/790) was gravimetrically made at the British Columbia Centers for Disease Control (BCCDC). Lyophilized standard was reconstituted in 0.5 ml of distilled water as per the package insert (National Institute for Biological Standards and Control) to yield a solution containing 105 IU/ml. By international agreement, 100,000 IU is defined as the amount of HCV in 1 ml of this standard (16, 17). Dilutions were made with anti-HCV and HCV PCR-negative human plasma. The dilution panel consisted of 15 replicates each (panel members) at concentrations of 1.0, 2.5, 5.0, 10, 20, 30, 40, 50, 60, and 70 IU/ml. Multiple aliquots of panel members were frozen at -70°C, and all aliquots underwent one freeze-thaw cycle.
Aliquots for HCV TMA testing were shipped on dry ice to the Bayer Reference Testing Laboratory (BRTL) in Emeryville, Calif. Specimens were assayed randomly with each of the panel members typically represented at least once in each run. Additional negative controls were added to each run of both HCV PCR and HCV TMA. HCV PCR was performed as directed by the manufacturer. For HCV PCR, the BCCDC employed the run validity criteria described in the package insert. BRTL assay procedures were followed for HCV TMA, a home-brew assay. Standard run validity criteria developed by GenProbe, the manufacturer of the assay, were employed by the BRTL. An assay run was considered valid if the controls gave correct results and a minimum of 90% of specimens had internal controls whose RLUs were above the value for the internal control cutoff. Specimen results were considered invalid if the internal control RLU value was below the internal control cutoff value.
Qualitative Cobas Amplicor PCR assay testing of the WHO panel. Over a period of 7 working days, six full runs and one partial run were carried out on a single Cobas instrument by the same BCCDC technologist, who was blinded to sample concentrations tested. Samples yielding gray zone results were retested in duplicate according to the manufacturer's directions.
Qualitative TMA-based testing of the WHO panel. Fifteen replicates of most of the dilution panel members (containing 1 to 50 IU/ml) were analyzed in two runs of 64 samples each. The 60-IU/ml and 70-IU/ml replicates were tested randomly on a subsequent assay run. In addition to the standard negative calibrators, four negative control samples were included in each run. Validity criteria for the run are described in the previous section on the WHO panel. All HCV TMA assays were performed on the same instrument and the same kit lot.
Assessment of assay limit of detection. The limit of detection, or analyte detection sensitivity, is defined as the minimum analyte concentration at which an assay can distinguish a positive specimen from negative specimens 95% of the time (13). A logistic regression model was used to estimate the detection rate as a function of concentration via the maximum-likelihood method. The limit of detection was estimated with the inverse of the estimated logistic equation to calculate the concentration corresponding to a 95% detection rate (8).
Clinical samples. Between December 1999 and February 2000, approximately 19,000 clinical specimens were received at the BCCDC for HCV testing. A subset of 300 specimens, as described below, were chosen for clinical evaluation of the nucleic acid test assays. Separate aliquots of these samples, whose serum had been removed from the clot within 4 h of the draw and stored at -70°C, had been used previously for HCV PCR testing. HCV PCR was initially performed because nucleic acid testing was requested or the specimen was anti-HCV indeterminate. Aliquots of all 300 samples were shipped on dry ice to the BRTL, where they were stored at -80°C prior to HCV TMA testing.
As per standard BCCDC procedures, specimens submitted for HCV testing are screened for anti-HCV by third-generation enzyme immunoassay (EIA) on an AxSYM instrument (Abbott, North Chicago, Ill.). Specimens with signals above the assay cutoff undergo secondary serological testing on an Ortho Vitros Eci instrument (Ortho-Clinical Diagnostics). Specimens whose EIA test signals are greater than 2 to 3 standard deviations (SD) above the cutoff of both assays have been shown to be serologically HCV positive >98% of the time (1, 2, 7, 9). When the results of one or both EIA tests are greater than the assay cutoff but the signal intensity is less than 2 SD above the assay cutoff, the specimen is considered serologically indeterminate.
Specimens were divided into three groups: 112 samples that were EIA indeterminate (signals < 2 SD above the cutoff) and for which HCV RNA was not detected by HCV PCR; 79 samples that were anti-HCV positive by both EIAs (> 2 SD above the assay cutoff) and for which HCV RNA was not detected by HCV PCR; and 105 samples that were positive by both EIAs (>2 SD above the cutoff) and for which HCV RNA was detected by HCV PCR. Three HCV PCR gray zone and one neonatal clinical specimen were also tested by HCV TMA. The PCR gray zone samples had insufficient volume to be retested in duplicate, as required by the package insert. The neonatal specimen had detectable HCV RNA by HCV PCR but had not been tested by EIA because antibody in neonates is passively acquired. Aliquots of all 300 samples were shipped on dry ice to the BRTL for HCV TMA testing.
Assessment of workflow. A data collection worksheet was developed for the HCV TMA and HCV PCR assays. One run of each assay was timed during the WHO panel testing. The HCV PCR workflow measurements were performed at the BCCDC, where one BCCDC technologist timed another BCCDC technologist performing one full run of 22 reportable specimens. The HCV TMA workflow assessment was performed at the BRTL, where one BRTL technologist timed a BCCDC technologist performing the first test run of 64 specimens as part of the WHO panel evaluation.
| RESULTS |
|---|
|
|
|---|
|
|
|
The hands-on and hands-off time as well as the total run time for one representative run of each assay were also determined (Fig. 4). The total run time was divided into time spent on (i) sample preparation, including RNA extraction and capture, and (ii) amplification, detection, and analysis. Hands-on time is defined as the time the technologist was actively manipulating the specimen or reagents, while hands-off time refers to incubation time periods when the technologist was not actively involved in assay procedures. For the HCV TMA assay run with 64 reportable results, the total run time was 5 h, 2 min, with a hands-on time of 2 h, 45 min. In contrast, for 22 reportable results, the HCV PCR assay run required 7 h, 45 min to complete, with a hands-on time of 4 h, 36 min.
|
| DISCUSSION |
|---|
|
|
|---|
The recent availability of the WHO standard facilitates interassay comparisons and allows more standardized assessment of performance, for example, limit of detection. Our data are consistent with the stated limits of detection for both HCV PCR and HCV TMA. Lee et al. (11) reported that HCV PCR had 100% sensitivity at 50 IU/ml and 91% sensitivity at 25 IU/ml. Our data, based on 95% sensitivity, showed the limit of detection for HCV PCR to be 45 IU/ml when gray zone specimens were not resolved and 32 IU/ml after specimens yielding gray zone results were resolved by retesting in duplicate.
Of note, when testing the WHO dilution panel on the HCV PCR, 20 of the 150 panel members gave gray zone results, and 16 of the 20 occurred when testing replicates at concentrations below 20 HCV IU/ml. For a given assay, a gray zone represents the range of analyte values that generate signals where there is an overlap between positive and negative test results (5). At a limiting dilution of the analyte, the ability of any assay to detect a target becomes a statistical process that is said to resemble a Poisson distribution where approximately one-third of the test reactions produce negative results (5). HCV PCR has a recognized gray zone and an algorithm developed by the manufacturer to resolve specimen status by retesting in duplicate. This study demonstrates that the resolution of HCV PCR gray zone specimens increased the sensitivity of HCV PCR from 45 to 32 IU/ml and that retesting in duplicate provides a statistical process to enhance the limit of detection of HCV PCR. It must be understood that using duplicate or triplicate testing to resolve specimens that yield gray zone signals can increase either assay sensitivity or specificity. This statistical approach to resolving gray zone specimens is possible because HCV PCR is highly specific (11).
Our data demonstrate that the limit of detection of the HCV TMA assay was 6 IU/ml, which is consistent with the 5-IU/ml limit of detection reported by Ross et al. (15) and Sawyer and colleagues (19). Published data have shown that HCV TMA is also very specific (15, 19). HCV TMA reports specimens as being either reactive (HCV RNA detected) or nonreactive (HCV RNA not detected). No gray zone value is reported. Given the 6-IU/ml limit of detection of HCV TMA, to evaluate whether HCV TMA has a gray zone would have required a thorough evaluation of test results from multiple replicates of a dilution series containing very low levels of HCV RNA (e.g., 5 to 0.5 IU/ml) (5). In order to verify whether specimens at such low dilutions of the WHO HCV standard truly contain HCV RNA, discrepant specimens would also need to be tested by an appropriately sensitive third-party assay. At this time, no standardized commercial HCV assay with sufficient sensitivity to adequately resolve discrepant results at such low HCV RNA concentrations has been described. Homebrew nested PCR assays which may be sensitive enough are prone to amplicon contamination and difficult to standardize (24).
The impact of resolving gray zone specimens on the laboratory workload depends on the frequency of specimens with low levels of HCV RNA. Within this highly selected specimen set, 3 of 300 specimens yielded initial gray zone results. Indeed, of the last 3,000 HCV PCR tests performed at the BCCDC, 1 to 2% yielded gray zone values on initial testing (data not shown).
To compare the performance of the two assays with clinical specimens, we selected subsets of specimens that are problematic to the laboratory along with appropriate controls. Specimen selection was influenced by analysis of the previous HCV testing results at the BCCDC. This laboratory performs approximately 75,000 anti-HCV tests per year, 8 to 10% of which have some degree of anti-HCV reactivity by AxSYM on initial screening. Of all anti-HCV-reactive specimens, 15 to 18% yield weak signals (<2 SD above the assay cutoff), and the other 82 to 85% yield signals greater than 2 SD above the assay cutoff (9). Without epidemiological data, it is difficult to determine the cause of weakly anti-HCV-reactive specimens (9, 14). Possible explanations include testing too early during acute infection (i.e., prior to complete seroconversion); blunted antibody responses due to immunosuppression; waning antibody due to resolved infection (21); and a false-positive serological test (9). Nucleic acid testing can determine if HCV RNA is present in these specimens. Similarly, in individuals with strongly reactive anti-HCV serology, nucleic acid testing is required to confirm active HCV infection. Confirmation of active infection is especially important when assessing individuals who undergo treatment and those with normal serum transaminase levels (10, 20).
We therefore assessed the concordance of the nucleic acid testing assays in selected specimen subsets, i.e., HCV PCR-negative, anti-HCV-indeterminate, and anti-HCV-positive specimens, to determine if the more sensitive HCV TMA could identify HCV RNA in additional specimens. Overall, the concordance of the two assays was excellent and varied between 96 and 100% depending on the combination of anti-HCV and HCV PCR results of the subsets tested. For example, all strongly anti-HCV-positive, HCV PCR-positive specimens were also HCV TMA positive. Of the 191 specimens (of 300) that were anti-HCV indeterminate or positive, HCV PCR-negative, four were HCV TMA assay positive. These specimens came from individuals with a history of substance abuse or from patients undergoing Rebetron therapy. Unfortunately, there was insufficient volume for further testing of these samples. Of note, Ross et al. (15) also reported the detection of HCV RNA by HCV TMA in four samples that had undetectable HCV RNA by Amplicor version 2.0 PCR among serial samples from 10 patients undergoing interferon alpha therapy. These were confirmed with a nonstandardized homebrew PCR assay.
For the three clinical specimens that gave initial gray zone results by HCV PCR and were negative by HCV TMA, there was insufficient specimen volume for retesting in duplicate as required by the HCV PCR diagnostic algorithm.
Assay sensitivity, throughput, and workflow all influence laboratory assay selection. While we ran 64 samples in our workflow study, the maximum throughput for HCV TMA (91 reportable samples) is more than four times that of HCV PCR (22 reportable samples). Notwithstanding the difficulty in accurately determining workflow times for assays that are incompletely automated, HCV TMA showed considerable timesaving over HCV PCR for both hands-on time and time to final result.
In summary, the HCV TMA assay demonstrated excellent concordance with HCV PCR, but it was more sensitive, required less time for test result completion, and had a greater throughput. Further studies in well-characterized populations will be required to properly assess the clinical impact of the increased HCV TMA sensitivity.
| ACKNOWLEDGMENTS |
|---|
We thank Kristina Whitfield (Posterdocs, Oakland, Calif.) for graphics.
| FOOTNOTES |
|---|
| REFERENCES |
|---|
|
|
|---|
This article has been cited by other articles:
| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Antimicrob. Agents Chemother. | Clin. Microbiol. Rev. |
|---|---|
| Clin. Vaccine Immunol. | ALL ASM JOURNALS |
|---|