ABSTRACT
The analytical performance of the Veris HCV Assay for use on the new and fully automated Beckman Coulter DxN Veris Molecular Diagnostics System (DxN Veris System) was evaluated at 10 European virology laboratories. Precision, analytical sensitivity, specificity, and performance with negative samples, linearity, and performance with hepatitis C virus (HCV) genotypes were evaluated. Precision for all sites showed a standard deviation (SD) of 0.22 log10 IU/ml or lower for each level tested. Analytical sensitivity determined by probit analysis was between 6.2 and 9.0 IU/ml. Specificity on 94 unique patient samples was 100%, and performance with 1,089 negative samples demonstrated 100% not-detected results. Linearity using patient samples was shown from 1.34 to 6.94 log10 IU/ml. The assay demonstrated linearity upon dilution with all HCV genotypes. The Veris HCV Assay demonstrated an analytical performance comparable to that of currently marketed HCV assays when tested across multiple European sites.
INTRODUCTION
Chronic hepatitis C virus (HCV) infection, affecting approximately 180 million people worldwide, can have long-term health outcomes ranging from minor histological liver changes progressing through to fibrosis, cirrhosis, hepatocellular carcinoma (HCC), and death (1–5). The goal of treatment is to prevent these adverse health outcomes. To evaluate treatment success, HCV monitoring by measuring HCV RNA viral load quantification in the blood is an essential tool. HCV RNA viral load measurements are recommended by current guidelines prior to treatment and posttreatment in order to determine treatment efficacy, as measured by sustained virologic response (SVR) at weeks 12 and 24 after the end of therapy (3, 6, 7). Additionally, viral load measurements during therapy can be used in patients with suspected antiviral resistance, as well as those at risk for poor compliance with a medication regime (8).
There are several assays currently on the market for the quantification of HCV viral load (VL) based on real-time PCR and branched DNA quantification (9–14). These methods are recommended based on their excellent analytical sensitivity, specificity, accuracy, and broad dynamic range of linear quantification. Many systems require separate instruments for extraction and amplification steps and batching of tests in order to optimize costs. Both can increase the time to result.
The Beckman Coulter DxN Veris Molecular Diagnostics System (DxN Veris System) is a new fully automated system for the quantitative analysis of molecular targets. The system integrates sample introduction, nucleic acid extraction, reaction setup, real-time PCR amplification, and detection using TaqMan chemistry, and result interpretation. The system allows true random access, introducing flexibility in the number of tests and number of assays that can be run at the same time. The Veris HCV Assay is an RNA-based quantitative nucleic acid amplification-based assay for HCV, calibrated to the 4th WHO international standard for hepatitis C virus (NIBSC 06/102 [15]) using the DxN Veris System.
The objective of this study was to evaluate the analytical performance of the Veris HCV Assay on the DxN Veris System at multiple sites in the European Union.
RESULTS
Precision. Table 1 shows the standard deviation (SD) for each component of the reproducibility evaluation, as well as the SD and percent coefficient of variation (% CV) total for each level tested. Panel 1 is not presented in the table, as all results returned a value of not detected (ND) (n = 269 [2 sites did not test panel 1]). The reproducibility evaluation expected a total of 40 measurements of each panel from each site. Some sites had fewer or greater numbers of measurements depending on several factors, including instrument down-time, technician vacations, local holidays, or extended testing over weekends. The results showed an SD of 0.22 log10 IU/ml or lower at each level tested and a % CV of 12.7% at the low end, decreasing to 2.9% at the high end. The mean and SD are in log10 international units per milliliter.
Precision of the Veris HCV Assay
Analytical sensitivity by LOD.Positive hit rates, with positive defined as either quantified (≥12 IU/ml) or detected but not quantified (<12 IU/ml), for the dilution panel for each site completing limit of detection (LOD) testing is presented in Table 2. The calculated LOD by probit analysis for each site is also presented in Table 2. For the 4th WHO standard (genotype 1a), the LOD, performed at one site, and determined by probit analysis was 9.0 IU/ml (95% confidence interval [CI], 7.7 to 10.9 IU/ml). For the WHO-traceable material (genotype 1a), the LOD, performed at 4 sites, ranged from 6.2 to 8.1 IU/ml (95% CI, 5.1 to 10.2 IU/ml).
Analytical sensitivity of Veris HCV Assay
Dilution linearity with patient samples for Veris HCV Assay
Specificity and performance with negative samples.All of the 94 samples from one site tested for specificity were not detected, yielding 100% clinical specificity, with a 95% confidence interval of 96.1 to 100.0%.
Of the total 1,089 negative samples tested for performance of the assay with negative samples from 6 sites, all returned results of not detected (n = 1,089).
Linearity and performance with HCV genotypes.A summary of the results for the dilution linearity using a high-titer sample performed at each of 8 sites are presented in Table 3. Linearity was demonstrated from 1.34 log10 IU/ml to 6.84 log10 IU/ml with clinical samples. R2was 0.99 at all sites. The maximum degree of nonlinearity was ±0.25 log10 IU/ml at all sites.
A summary of the results for performance with HCV genotypes for the combined data from 3 sites is shown in Table 4. The maximum degree of nonlinearity was ±0.17 log10 IU/ml for all HCV genotypes. Figure 1 shows the regression plots for each HCV genotype, 2 to 6.
Performance with HCV genotypes for Veris HCV Assay
Linear regression plots for Beckman Coulter-supplied dilutions of HCV genotypes 2 to 6. Combined data from 3 testing sites with plots showing observed result (in log10 international units per milliliter) versus expected result (log10 international units per milliliter).
DISCUSSION
In this study, we evaluated the analytical performance of the Veris HCV Assay for use on the novel DxN Veris System, a new fully automated system for the quantitation of molecular targets. Performance was evaluated for precision (reproducibility), limit of detection (analytical sensitivity), negative samples, linearity, and HCV genotypes at a total of 10 European clinical virology laboratories.
The precision of the Veris HCV Assay showed an SD of 0.22 log10 IU/ml or lower for each level tested, with the % CV decreasing from 12.7% at the low end to 2.9% at the high end. The SDs and % CVs were comparable with those seen with other approved HCV assays (9–14). In this evaluation, within-run, between-lot, and between-site variability contributed most to the total imprecision seen. For between-lot calculations, only variability between different assay reagent pack (ARP) lots (4 total) and calibrator lots (2 total) was accounted for, while variability for different extraction purification cartridge B (EPB) lots was not. Additional analysis not shown here indicated that the different calibrator lots contributed very little to the variability between lots, with the majority related to ARP lots. As multiple EPB lots were also used at each site, this may have increased the variability, but between-lot variability would be confounded with between-site variability.
The limit of detection performed on HCV genotype 1a and determined by probit analysis was 9.0 IU/ml with the 4th WHO Standard and 6.8 to 8.0 IU/ml with WHO-traceable material. Results from a study done as part of the clinical evaluation submission showed that the Veris HCV Assay was able to detect 95% of replicates for all major HCV genotypes to an LOD of 12 IU/ml or lower (data not shown). These LOD results are comparable to that of the Roche Cobas TaqMan HCV test, version 2.0, with a claimed LOD of 15.0 IU/ml (16), and the Abbott RealTime HCV assay, with a claimed LOD of 12.0 IU/ml (17). A broad linear range and detection of all HCV genotypes were also demonstrated.
These results show that the Veris HCV Assay performance is in line with that seen for other currently marketed real-time PCR molecular assays (9–14) and fulfills the recommendations in current clinical practice guidelines for predicting a patient achieving SVR at 12 and 24 weeks posttreatment. Results were generated across multiple sites and showed comparative overall precision and consistently low LOD despite the introduction of increased variability related to the use of multiple instruments, lots, operators, and preanalytical techniques, such as sample processing, handling, and storage prior to testing. Preanalytic technique differences from laboratory to laboratory make it necessary for each to be aware of potential sources of variability when running a new system. Many currently marketed PCR systems require the use of separate instruments for extraction/purification and PCR amplifications steps, as well as batching of tests. The DxN Veris System for HCV monitoring is the first PCR system to integrate sample introduction, nucleic acid extraction, reaction setup, real-time PCR amplification, and detection using TaqMan chemistry and result interpretation. The system also provides true random access, allowing laboratories the ability to run as little as a single test per assay without having to batch samples.
MATERIALS AND METHODS
The performance characteristics of the Veris HCV Assay are summarized in Table 5. A total of 10 European sites participated in this evaluation. Sample panels provided by Beckman Coulter were prepared from the 4th WHO international standard for HCV (15) or WHO-traceable HCV material (AcroMetrix; Thermo Scientific, Fremont, CA, USA) and were shipped on dry ice and stored at −70°C until testing. Samples provided by the sites were leftover patient samples processed per routine and tested fresh or stored at −70°C until testing. All samples were K2EDTA-plasma. Each site had ethics approval prior to the start either for the use of leftover patient samples, or a waiver for the use of such samples. All sites followed the same protocol, that of the DxN system HCV assay beta study.
Summary of Veris HCV Assay characteristics
Calibration was required for all testing and was performed once prior to start at each site. Each site used only one lot of assay reagent packs (ARPs) and calibrators during the entire study, but the same lot was not used at all sites. Overall, four lots of ARPs and two lots of calibrators were used. Multiple lots of extraction purification (EP) cartridge B reagents were used at each site. Quality controls (negative, low, and high) were passed daily prior to the start of testing.
Precision.To assess precision, Beckman Coulter provided each site with the same HCV standard dilution series panel prepared from WHO-traceable HCV material (genotype 1a). The series contained five K2EDTA-plasma samples with the following approximate concentrations of the HCV RNA virus: 0.0, 36 IU/ml (1.56 log10 IU/ml), 2,400 IU/ml (3.38 log10 IU/ml), 100,000 IU/ml (5.00 log10 IU/ml), and 80,000,000 IU/ml (7.9 log10 IU/ml). Nine sites performed precision testing for the Veris HCV assay. Each site performed testing in one run per day, with each panel member tested in duplicate in each run, for 20 days, for a total of 40 replicates per panel member per site. The study design was based on Clinical and Laboratory Standards Institute (CLSI) EP05-A3 and EP15-A3 guidelines (18, 19). Within-run, between-run, within-day, between-day, between-lot (ARPs and calibrators), between-site, and total imprecision were evaluated for each level tested using nested analysis of variance on the log10-transformed HCV RNA values. The standard deviation (SD) was calculated for all components of variability, and the coefficient of variation (CV) was calculated only for total variability.
Analytical sensitivity.To assess the limit of detection (LOD), Beckman Coulter provided each testing site with the same HCV standard dilution series panel containing five K2EDTA-plasma samples with concentrations of the HCV RNA virus around the LOD. The levels were 0.0 IU/ml, 2.0 IU/ml, 6.0 IU/ml, 10.0 IU/ml, and 20.0 IU/ml prepared with 4th WHO international standard (1 site) or WHO-traceable material (4 sites). Both were HCV genotype 1a. Five sites performed LOD testing. Each site performed LOD testing in one run per day, with each panel member tested in 12 replicates, for a total of 3 days, for a total of 36 replicates per panel member per site. The study design was based on the CLSI EP17-A2 guideline (20). The concentration at 95% positive results was determined using probit analysis.
Specificity and performance with negative samples.To assess specificity, one site provided 94 unique K2EDTA-plasma specimens that were negative for HCV RNA by a comparator assay and HCV negative by immunoassay. The proportion not detected (ND) with the 95% CI using the Wilson score method was determined. Performance with negative samples was assessed at 6 sites using both Beckman Coulter and site-provided samples. Beckman Coulter samples, tested at 5 sites, were made from a single lot of pooled K2EDTA-plasma samples taken from 60 subjects (BioreclamationIVT, NY). Site samples were those used for clinical specificity from one site, tested in duplicate. All specimens were negative for HCV RNA by a comparator assay and HCV negative by immunoassay. The number (n) and percent detected and not detected (ND) were determined.
Linearity and performance with HCV genotypes.To assess linearity, sites selected a high-titer HCV patient sample and performed 4 to 6 serial dilutions with known negative patient sample or Basematrix down to below the LOD. Each level (original undiluted sample and dilutions to below LOD) was tested in 4 replicates. The study design was based on the CLSI EP06-A guideline (21). The linear regression line, R2, root mean square error (RMSE), and maximum difference from linearity were determined.
To assess performance with HCV genotypes, Beckman Coulter provided 3 testing sites with an HCV standard dilution series panel prepared from WHO-traceable material, containing four to five K2EDTA-plasma dilution samples from 5 HCV genotypes (2 to 6). The sites tested each sample dilution from each genotype in triplicate (2 sites) or quadruplicate (1 site), for a total of 10 replicates per level. The linear regression line, maximum difference from linearity, R2, and RMSE were determined.
ACKNOWLEDGMENTS
We thank Beckman Coulter for technical support (Dominique Philbert) and field service engineers (France, Laurent Alix; Germany, Michael Kistner; Italy, Nevio Borra; Spain, Roman Lopez Sanchez; and UK, Stuart Ross) and application specialists (France, Emmanuel Gay; Germany, Ingo Boettcher; Italy, Peter Ehrenheim; Spain, Carelia Garcia; UK, Brendan McKeown) who worked diligently on this project. We also thank Sam Douthwaite, who served as principal investigator at the London site, as well as the following laboratory technicians for their hard work: Elise Belloc (Bordeaux), Brigette Tauzin (Bordeaux), Catherine Hasle (Toulouse), Jérôme Boineau (Toulouse), Emanuela Franchetti (Milan Niguarda), Davide Mileto (Milan L. Sacco), Alessandro Mancon (Milan L. Sacco), Gudrun Naeth (Aachen), Bianca Haase (Aachen), Patricia de Molina (Barcelona), Paquita Gutierrez (Madrid), Sagrario Zurita (Madrid), Alexandra Yates (Sheffield), Lucy Neil (Sheffield), and Anastasia Papadakis (London). Thank you to Tamra Roddy, Beckman Coulter Carlsbad, CA, USA, for work on the statistical analysis.
P.B. and R.D. have consulted for Beckman Coulter. P.B., R.D., D.F., H.F., J.H., J.I., and D.W. have received speaking honoraria. P.B., R.D., D.F., H.F., J.H., J.I., M.A.M., J.R., and D.W. received paid travel to conference.
This study was supported and funded by Beckman Coulter, Inc. D.W.R. and A.A. are Beckman Coulter employees.
The Veris HCV Assay and the DxN Veris System are CE marked and not currently for sale or distribution in all markets, including the United States. The DxN Veris System is also known as Veris Molecular Diagnostics System and Veris MDx System.
FOOTNOTES
- Received 2 November 2016.
- Returned for modification 28 November 2016.
- Accepted 25 January 2017.
- Accepted manuscript posted online 1 February 2017.
- Copyright © 2017 American Society for Microbiology.
