ABSTRACT
Assessment of the viral load in hepatitis C virus (HCV) genotype 1-infected patients is critical before, during, and after antiviral therapy. In patients achieving a rapid virological response at week 4 of treatment, the viral load at the baseline is considered a predictive criterion of a sustained virological response 24 weeks after the discontinuation of treatment. A ≥2-log10 drop in the viral load at week 12 of treatment (early virological response) triggers the continuation of therapy. We organized a multicenter study (MS) for diagnostic laboratories involved in the quantification of HCV RNA. Commercial assays, including two based on real-time reverse transcription-PCR (TaqMan system), and in-house methods, were used by the 61 participants. The overall reproducibility of the commercial quantitative nucleic acid amplification techniques (qNAT) was acceptable. As the intermethod variability among commercial qNAT for HCV RNA was still present, the manufacturers of these test kits should join efforts to harmonize the means of quantification of HCV RNA. This study also shows that caution should be exercised when the baseline viral load is evaluated and when the 2-log10 reduction after 12 weeks of therapy is interpreted. Finally, this MS confirms the higher sensitivity of the commercial qNAT based on the TaqMan system, making them the elective assays for the monitoring of therapy.
Combination treatment with peginterferon alfa and ribavirin is currently considered the therapy of choice for patients chronically infected with hepatitis C virus (HCV) (5). As HCV genotype 1-infected patients are less responsive to this therapy, especially in the presence of a high virus titer, assessment of the viral load before, during, and after antiviral therapy plays a crucial role in the management of these patients (6). The viral load at the baseline is deemed important in HCV genotype 1-infected patients achieving a rapid virological response (RVR), defined as an undetectable serum HCV RNA load (<50 IU/ml) by a sensitive molecular assay at week 4 of treatment. In two studies (9, 15), a baseline level of viremia of <400,000 IU/ml in patients with an RVR, treated for only 24 weeks as opposed to the canonical 48 weeks, was shown to be a criterion predictive of achieving a sustained virological response (SVR), defined as the absence of serum HCV RNA (<50 IU/ml) by a sensitive molecular assay 24 weeks after the cessation of therapy. The viral load at week 12 of treatment was shown to have an impact on the outcome of treatment in HCV genotype 1-infected patients, as a failure to obtain a ≥2-log10 drop in the HCV RNA level, namely, an early virological response (EVR), was strongly associated with nonresponsiveness, i.e., a failure to achieve an SVR (4). Consequently, in the absence of an EVR, the treatment is considered ineffective for these patients and can be discontinued.
The quantification of HCV RNA requires easy, reliable, and standardized tests with high-level of reproducibility in order to be used in the clinical routine (2, 10). Since the establishment in 1998 of the World Health Organization (WHO) HCV international standard, expressed in international units (12), the standardization of different assays based on quantitative nucleic acid amplification techniques (qNAT) has become feasible. Commercial qNAT based on two different technologies, i.e., noncompetitive reverse transcription (RT)-PCR and analysis of branched DNA (bDNA), were initially developed. More recently, real-time RT-PCR assays, based on the TaqMan system, were introduced on the market. Compared to the classical qNAT, this system has proved to be more precise, to have a broader range of quantification and to be more sensitive, with have lower limits of detection (10 to 15 IU/ml).
Despite the adoption of a common unit of measurement, differences in the performances of these qNAT have been observed, mainly due to the different technologies used and to the different efficiencies of HCV RNA hybridization to complementary probe and primer sequences (3, 14). The latter technical aspect may lead to method-related discrepancies in the quantification of the viral load. The intrinsic variability in the efficiency of qNAT is also accompanied by the day-to-day variability due to extrinsic factors (e.g., operators, instruments, and sample processing). The introduction of systems that prevent carryover contamination (uracil N-glycosylase) has reduced the percentage of false-positive results. Automation of the qNAT has also contributed to this reduction in cross-contamination, with the additional benefit of minimizing the intra- and interassay variabilities.
For diagnostic laboratories, participation in a multicenter study (MS) represents a critical tool for the assessment of their performance. With respect to the quantification of HCV RNA by NAT, up to now only a few MSs have been done and none included the use of commercial assays based on the TaqMan system (1, 8, 11, 13).
To meet the demand for these MSs from diagnostic laboratories, we organized a new study for the quantification of HCV RNA by NAT. The state of the art of all the available commercial qNAT, in terms of the detection limit and the dynamic range, were taken into account in the development of the panels. Furthermore, the design of the study took into consideration the algorithms mentioned above for the use of antiviral therapy in HCV genotype 1-infected patients.
MATERIALS AND METHODS
Negative and positive samples.Negative samples were prepared by using a pool of plasma made up of 20 donated plasma samples negative for HCV, human immunodeficiency virus, and hepatitis B virus by serological and NAT testing.
Two groups of positive samples were prepared from an anti-HCV and HCV RNA (genotype 1b)-positive donation tentatively calibrated by the Istituto Superiore di Sanità (ISS) against the WHO HCV RNA international standard (IS 96/798) by a qNAT (in-house real-time RT-PCR) and found to have a presumed titer of 6.69 log10 IU/ml. This donation was diluted with the negative plasma pool mentioned above, as follows: (i) 1:15 for the high-titer samples (HTS), chosen to have a final titer within the range of linearity of the available commercial qNAT for HCV RNA, and (ii) 1:1,500 for the low-titer samples (LTS), to simulate the minimal viral load decrease considered to be predictive of a positive response to antiviral treatment.
Positive samples with HCV at a concentration of 50 IU/ml were prepared by appropriately diluting the WHO HCV IS 96/798 (genotype 1).
A total of 1,600 vials, including negative samples and positive samples (HTS, LTS, samples containing 50 IU/ml), were prepared and stored at −80°C and were sequentially numbered from 0001 to 1600. The negative or positive status of the samples was confirmed by the ISS. To this aim, the vials to be thawed and tested were randomly selected.
qNAT.Four commercial qNAT for the detection of HCV RNA were used in this MS: the Cobas TaqMan assay (CTM) with automated extraction (CAP/CTM) or manual extraction (HP/CTM) (Roche Diagnostics GmbH, Mannheim, Germany); the Cobas Amplicor Monitor assay (CAM; Roche Diagnostics GmbH); the RealTime HCV assay (RealT; Abbott Laboratories. Abbott Park, IL); and the Versant HCV RNA assay version 3.0 (bDNA; Siemens Healthcare Diagnostics, Deerfield, IL). These commercial qNAT are listed in Table 1, along with their respective detection limits and dynamic ranges, as stated by the kits' manufacturers. Four independent in-house assays (INH) were also used in this MS, and they were all grouped together. Two of these assays were completely developed in-house, while the remaining two were a combination of commercial test kits, one for extraction (Qiagen column) and one for amplification (Arthus kit).
Commercial qNAT used in this MS
Study design and compositions of panels.Each participant was sent two panels identified with a code number that was randomly assigned (e.g., A1, C2). Each of the two panels was made up of eight samples: three HTS, three LTS, one 50-IU/ml sample, and one negative sample. All of them were randomly selected. According to the testing protocol, the participants had to test the panels in two separate runs, 1 to 2 weeks apart, and to report the results in IU/ml. At the end of the study, each participant was due to receive a detailed technical report with all the results.
Participants.Fifty-eight Italian diagnostic laboratories, one Spanish diagnostic laboratory, and two kit manufacturers (Roche and Abbott) took part in this study, for a total of 61 laboratories. A coded number identified each laboratory throughout the study.
Shipment.The panels were shipped on dry ice. The participants were asked to check the integrity of the parcel, the presence of dry ice, and the frozen status of the samples and to fax this information to ISS by using the Acknowledgment of Receipt form.
Analysis of results for HTS and LTS.The IU/ml titers for the HTS and LTS reported by the participants were converted to log10 titers. These values were then separately used for the following calculations. (i) For each laboratory, the respective geometric mean (GM) per laboratory, expressed in log10 IU/ml [GM(log10)], and the standard deviation (SD) were calculated for each panel [GM(log10)panel ± SD; HTS, n = 3; LTS, n = 3] and for the two panels together [GM(log10)panel 1 + 2 ± SD; HTS, n = 6; LTS, n = 6]. A consensus mean titer (CmeanT) and the 95% confidence interval were then established for the HTS and LTS by using all the log10 titers obtained in the study.
(ii) These GM(log10)panel ± SD values were then grouped by qNAT for each run, and thus the GM(log10) for each run of each assay [GM(log10)assay] ± SD was obtained. To evaluate the performance of the different qNAT used in this study, a range of acceptable values equal to GM(log10)assay ± 0.15 was established by taking into account the SDs of all the commercial assays. For each qNAT, Grubb's test (7) was used to detect outliers. The overall GM(log10) [GM(log10)overall] ± SD for each qNAT and the coefficient of variation (CV) were then calculated for the HTS and LTS.
(iii) The differences in the titers between the first and the second runs and vice versa were calculated for the HTS and LTS.
RESULTS
Overall, 130 panels were distributed to the 61 laboratories that participated in the study. Four laboratories tested the panels using a second qNAT, for a total of 65 data sets.
All participants completed the study and returned the reporting form. Testing of the two panels was finalized within 20 to 45 days from the time of receipt of the panels. No deviations from the protocol were observed, with the exception that one of the four INH users reported the results in as the number of copies/ml instead of the number of IU/ml. Consequently, the results provided by that laboratory were not included in the statistical analysis.
The GM(log10)panel 1 + 2 ± SD values, grouped by qNAT, are plotted in Fig. 1 for HTS and LTS. As no statistically significant differences were observed between the CAP/CTM and the HP/CTM assays, these results were analyzed together under CTM. All the GM(log10)panel 1 + 2 values for bDNA and RealT fell within the respective range of acceptability [GM(log10)assay ± 0.15] for both HTS and LTS. With respect to the other qNAT, a total of seven GM(log10)panel 1 + 2 values falling outside the range of acceptability were observed for both HTS (three by CTM, two by CAM, and two by INH) and LTS (four by CTM, one by CAM, and two by INH).
For each laboratory, the GM(log10)panel 1 + 2 ± SD values, expressed in log10 IU/ml and grouped by qNAT, are reported for the HTS and LTS. Each point represents the mean ± SD. For each group, the respective GM(log10)assay value is also reported (solid line) with the established ±0.15 range of acceptability (dashed lines).
Titers for HTS.For the 390 samples tested, 386 quantitative HCV results were reported by the participants, as the results for three samples from CTM users were invalid and the result for one sample from a CAM user was aberrant. The latter result was due to a mistake in the sample predilution phase and was not included in the statistical analysis. The GM(log10)assay ± SD values along with the percentages of individual results (n = 1) falling within the range of acceptability of each qNAT are reported in Table 2 for each run.
Intra-assay precision for HTS and LTS, grouped by qNAT, for runs 1 and 2
The GM(log10)overall ± SD values and the respective CVs, calculated for each qNAT, are reported in Table 3, along with the CmeanT value (5.62 log10 IU/ml ± 0.16; 95% confidence interval, 5.58 to 5.67 log10 IU/ml).
GM(log10)overall ± SD values and CVs for HTS and LTS
Titers for LTS.For the 390 samples tested, 388 quantitative HCV results were reported by the participants, as 2 invalid results were reported by CTM users. The GM(log10)assay ± SD values along with the percentages of individual results (n = 1) falling within the range of acceptability of each qNAT are reported in Table 2 for each run.
The GM(log10)overall ± SD values and the respective CVs, calculated for each qNAT, are reported in Table 3, along with the CmeanT value (3.61 log10 IU/ml ± 0.19; 95% confidence interval, 3.56 to 3.66 log10 IU/ml).
Two-log10 difference in HCV RNA levels between HTS and LTS.All the possible combinations between HTS and LTS in the two panels (HTSpanel 1-LTSpanel 2 and HTSpanel 2-LTSpanel 1) were taken into account for each participant, and a total of 1,097 differences were obtained. The results provided by the INH users were excluded from this evaluation due to the high degree of variability of the GM(log10)assay values (see above). For each assay method, the mean for these combinations was very close to the expected value of 2.0 log10, with the SDs varying from 0.09 to 0.17 (Table 4).
Two-log10 difference in HCV RNA between HTS and LTS
Samples with 50 IU/ml.Only the 30 CTM and RealT users were expected to correctly identify the 130 samples with HCV at 50 IU/ml on the basis of the detection limit of those qNAT, for a total of 60 positive results. Instead, 58 positive results were provided by these participants, as one CTM user failed to detect HCV in the sample with virus at 50 IU/ml in both panels (data not shown). Although 50 IU/ml was below the detection limit of the assays, one CAM user and five bDNA users reported positive results for samples from one of the two panels with HCV at 50 IU/ml.
INH users were not capable of detecting the 50 IU/ml samples with the exception of one laboratory (using a combination of two commercial kits) that correctly identified one of them.
Negative samples.Finally, 129 results for 130 negative samples (1 result from a CTM user was invalid) were reported by the participants. Five bDNA users reported positive results for these samples. Four of these users were the same ones that reported a positive result for the 50-IU/ml samples.
DISCUSSION
This study reports the results of an MS for the quantification of HCV RNA in which the participants used in-house NAT and commercial qNAT, including two assays based on a real-time RT-PCR (TaqMan system) that were recently introduced on the market. Quantification of the HCV RNA load in genotype 1-infected patients is critical before, during, and after antiviral therapy. To simulate the 2-log10 load drop in the viral load in patients after 12 weeks of antiviral therapy, which triggers the continuation of treatment, samples with a 2-log10 difference in HCV RNA levels (HTS and LTS) were included in two panels that were sent to each participating laboratory to be tested in two separate runs.
Of the 61 laboratories participating in the study, 57 used a commercial qNAT (CAM, CTM, RealT, or bDNA), while 4 used INH. The results from one of the INH users were excluded due to the reporting of the results as the number of copies/ml rather than the number of IU/ml. For the HTS, the precision was quite variable among the qNAT (Table 2). This appeared to be especially true for INH, CTM, and CAM, which, consequently, gave lower percentages of results within the range of acceptability compared with the percentages of results within the range of acceptability obtained by RealT and bDNA (22 to 78% and 99 to 100%, respectively) (Table 2). However, all methods, with the exception of the INH, showed satisfactory reproducibility (CV < 3.0%); INH had a CV of 5.9% (Table 3). In particular, the worst CV was exhibited by the INH that was completely developed in-house. In reference to the mean titers obtained by the four commercial qNAT, differences of up to 0.33 log10 IU/ml were observed (for CTM versus bDNA). On the basis of this finding, a blood viral load of approximately 850,000 IU/ml obtained by CTM would correspond to a load of 400,000 IU/ml by bDNA. In HCV genotype 1-infected patients achieving an RVR, this discrepancy could influence the evaluation of the therapy's outcome (an SVR). Therefore, a clinician should always take into account the qNAT used by the laboratory when evaluating the viral load at the baseline. Also in the case of LTS, lower percentages of results within the range of acceptability were observed for INH, CTM, and CAM compared with the percentages of results within the range of acceptability observed for RealT and bDNA (33 to 79% and 93 to 97%, respectively) (Table 2). The precision was more variable for LTS than for HTS, as none of the assays showed 100% of the results within the range of acceptability. Furthermore, all methods showed a somewhat less satisfactory reproducibility (CV < 5.0% for LTS versus CV < 3.0% for HTS) with a peak CV of 10.9% for INH (Table 3). Once again, the worst CV was exhibited by the INH completely developed in-house. Differences in the mean titers for LTS were observed among the four commercial qNAT as well (up to a 0.38-log10 IU/ml difference between CAM and bDNA). Considering the precision of the results obtained, the range of acceptable values established in this MS, namely, GM(log10)assay ± 0.15, proved to be more appropriate than the one used in previous studies, namely, GM(log10)assay ± 0.50.
HTS and LTS were included in both panels in order to simulate testing of HCV genotype 1-infected patient samples taken at the beginning of antiviral therapy and after 12 weeks of antiviral therapy. This design allowed us to evaluate the ability of the diagnostic laboratories to detect a 2-log10 difference in the HCV RNA load. It is worth noting that in this simulated scenario, a high percentage (∼40%) of all the 1,097 differences were <2.0 log10 units. In clinical practice, this would lead to the incorrect discontinuation of therapy even in the presence of a real 2-log10 drop in the viral load. Therefore, the qNAT used for the quantification of HCV RNA in clinical samples should also be taken into consideration when the efficacy of the therapy is being evaluated. For example, if in the simulated scenario mentioned above we assume an uncertainty of 2 SDs, the percentage of unacceptable values (<2.0-log10 decrease in the viral load) decreases from ∼40% to ∼2% (Table 4). On the basis of these observations, patients should be monitored by the same qNAT for the entire cycle of antiviral therapy.
In patients achieving an RVR and an SVR, the blood is expected to contain undetectable amounts of viral RNA when a sensitive test with a lower limit of detection of 50 IU/ml is used (10). Therefore, to assess the ability of diagnostic laboratories to detect low levels of HCV RNA, samples with HCV at 50 IU/ml were included in the panels. This concentration is close to the lowest detection limit of most commercial assays, namely, threefold the CTM and RealT values and just below the detection limits of bDNA and CAM. As expected, 97.7% of the laboratories using CTM and RealT correctly quantified the virus in the samples with virus at 50 IU/ml. With respect to the bDNA and CAM users, six of them found one or more of these samples to be positive. As the detection limits of bDNA and CAM are much higher than 50 IU/ml (615 and 600 IU/ml, respectively) and the titers reported by those laboratories were much higher than 50 IU/ml (>700 IU/ml), this strongly suggests that these were false-positive results due to either cross-contamination or carryover events.
At the end of this MS, we were able to quantify the virus in the HTS and LTS included in the panels. On the basis of preliminary testing by ISS and by two kit manufacturers (Roche Diagnostics and Abbott Molecular Inc.), provisional titers of 5.68 log10 IU/ml and 3.65 log10 IU/ml, respectively, were assigned to these samples. From the overall mean of the individual data received from the participants, consensus titers of 5.62 log10 IU/ml and of 3.61 log10 IU/ml were calculated for the HTS and LTS, respectively, satisfactorily confirming the provisional titers that were established.
In conclusion, the results of the present study indicate that the commercial qNAT currently used are satisfactory for the quantification of HCV RNA. In fact, the overall reproducibility, which is slightly better for HTS than for LTS, proved to be acceptable (CV < 5%). In particular, regarding the two qNAT based on the TaqMan system, CTM and RealT, the latter showed a greater precision (CV = 1.06 to 2.34%) than the former (CV = 2.41 to 4.04%), although there could be a bias due to the smaller number of RealT users. It is worth noting that despite the establishment of the WHO HCV international standard, in which the viral content is expressed in IU/ml, the intermethod variability among commercial qNAT for the detection of HCV RNA is still present. Therefore, it would be advisable for the manufacturers of these kits to join efforts to harmonize the means of quantification of HCV RNA. Such a collaboration is particularly desirable in light of the clinical relevance of the qNAT. This study also shows that caution should be exercised both when the baseline viral load is being evaluated and when the 2-log10 reduction in the viral load after 12 weeks of therapy is being interpreted. Finally, this MS confirms the higher sensitivities of the commercial qNAT based on the TaqMan system (CTM and RealT), consistently capable of detecting low levels of HCV RNA, which makes them the elective assays for the establishment of an RVR and an SVR.
ACKNOWLEDGMENTS
We are grateful to all participating laboratories and manufacturers for their contribution to this HCV Multicenter Study: IRCCS, Aviano, Italy; UO Igiene Policlinico, Bari, Italy; Policlinico, Bari, Italy; Ospedale Policlinico Consorziale, Bari, Italy; Azienda Ospedali Riuniti, Bergamo, Italy; Ospedale Maggiore, Bologna, Italy; ASL 13, Borgomanero, Italy; Spedali Civili, Brescia, Italy; AUSL, Cesena, Italy; Azienda Ospedaliera, Cosenza, Italy; Azienda Istituti Ospitalieri, Cremona, Italy; Ospedale Maggiore, Fossano, Italy; PoloH1 ASL RMH, Frascati, Italy; Azienda Ospedaliera, Desenzano Garda, Italy; Ospedale San Martino, Genoa, Italy; ASS 2 Isontina, Gorizia, Italy; Ospedale, Lanciano, Italy; ASL 5 Ospedale S. Andrea, La Spezia, Italy; Ospedale Santa Maria Goretti, Latina, Italy; Azienda Ospedaliera, Lodi, Italy; Ospedale Campo di Marte, Lucca, Italy; Azienda Ospedaliera Carlo Poma, Mantua, Italy; Ospedale Niguarda, Milan, Italy; Azienda Ospedaliera San Paolo, Milan, Italy; Ospedale Sacco, Milan, Italy; IRCCS Multimedica, Milan, Italy; Ospedale San Raffaele, Milan, Italy; Policlinico, Modena, Italy; Azienda Ospedaliera Maggiore, Novara, Italy; Azienda Ospedaliera, Padua, Italy; Ospedale Civile AUSL 14, Padua, Italy; Istituto di Igiene Università, Palermo, Italy; IRCCS Policlinico, Pavia, Italy; Servizio Immunotrasfusionale, Pescara, Italy; Ospedale Civile, Portogruaro, Italy; Ospedale Pertini, Rome, Italy; Policlinico Umberto I, Rome, Italy; Azienda Ospedaliera San Camillo-Forlanini, Rome, Italy; Ospedale San Giacomo, Rome, Italy; Ospedale San Filippo Neri, Rome, Italy; Ospedale Spallanzani, Rome, Italy; Laboratorio Virol Molec PTV, Rome, Italy; Ospedale G. Fracastoro, San Bonifacio, Italy; Azienda USL 1, Sassari, Italy; ASL SA1, Scafati, Italy; Policlinico Le Scotte, Siena, Italy; Azienda Ospedaliera Valtellina, Sondrio, Italy; Ospedale San Giovanni Battista, Turin, Italy; Ospedale Amedeo Savoia, Turin, Italy; Azienda Ospedaliera, Treviglio, Italy; Azienda Ospedaliera SM Misericordia, Udine, Italy; Policlinico Universitario, Udine, Italy; Ospedale Circolo Fondazione Macchi, Varese, Italy; ASL 14 Presidio Ospedaliero, Verbania, Italy; Ospedale Sant'Andrea, Vercelli, Italy; ULSS 16, Vicenza, Italy; Centro de Transfusion Galicia, Spain; Roche Diagnostics GmbH, Germany; Abbott Molecular Inc., United States; Abbott S.r.l., Rome, Italy; and Siemens Healthcare Diagnostics, Milan, Italy.
We are also grateful to Katia Colombo and Cristina Marra for their support in the organization of the MS.
FOOTNOTES
- Received 16 March 2009.
- Returned for modification 29 May 2009.
- Accepted 10 July 2009.
- Copyright © 2009 American Society for Microbiology
