Research and Development, Gen-Probe Incorporated, San Diego, California 92121
Received 25 May 2001/ Returned for modification 12 November 2001/ Accepted 13 April 2002
| ABSTRACT |
|---|
|
|
|---|
99.5% in both normal donor specimens and plasma containing potentially interfering substances or other blood-borne pathogens. Statistical receiver operating characteristic plots of 1 - specificity versus sensitivity data determined very wide analyte cutoff values for each assay at the point at which the assay specificity and sensitivity were both
99.5%. The sensitivity, specificity, and throughput capability predict that these assays will be valuable for large-volume plasma screening, either in a blood bank setting or in other diagnostic applications. | INTRODUCTION |
|---|
|
|
|---|
As of the end of 1999, 733,374 cases of AIDS had been reported to the Centers for Disease Control and Prevention (CDC), and through June 2000, 438,795 people had died of AIDS in the United States. AIDS prevalence continues to rise, with 320,000 persons living with AIDS in the United States in 1999 (CDC Divisions of HIV/AIDS Prevention Commentary, www.cdc.gov/hiv/stats 12, 2000). The prevalence of HIV-positive blood donations in first-time donors (first-time donors represent 21.1% of all donations) noted by the American Red Cross is 23 per 100,000. Among repeat donors (78.9% of all donations), this prevalence is 3 per 100,000, and the weighted overall prevalence is 7 per 100,000. The window of infectivity before seroconversion is approximately 22 days but may be up to 6 months in some cases. It is estimated that 1.5 HIV-1 infectious units per million donations escape detection with the licensed serological assays currently in place (8, 15).
Currently, detection of HIV-1 infection in the blood bank and clinical settings is based on serologic detection of p24 antigen and antiviral antibodies by enzyme immunoassay. Confirmation of antibody-positive samples is done by supplemental antibody tests such as Western blot or immunofluorescence assays, while confirmation of p24 positivity is done by neutralization. Follow-up of a positive result typically involves antiviral treatment and monitoring of the HIV-1 RNA titer by quantitative HIV-1 assays (2, 3, 15). Although the sensitivity of HIV antibody tests has increased and sensitive tests for p24 antigen have been developed and implemented, a window period still exists between infection and the appearance of detectable serological markers. Addition of the p24 antigen test has reduced the average window period from 22 days to approximately 16 days, since detectable antigenemia generally precedes seroconversion (2, 15). However, p24 antigen tests have a high false-positive rate and are less sensitive than nucleic acid-based amplification testing (NAT) (36). HIV-1 p24 antigen screening of blood donations, implemented in 1996, has only yielded six cases of antigen-positive, HIV-1 antibody-negative donors within more than 24 million donations screened (37). Several studies suggest that use of NAT could reduce the window period by 6 to 11 days, further reducing the risk of acquiring HIV from blood transfusions (7, 6, 9, 15, 19, 30).
At least 40% of chronic liver disease is related to HCV, resulting in an estimated 8,000 to 10,000 deaths each year (CDC data). The overall seroprevalence of HCV in the United States is 3.9 million (1.8%) (CDC National Center for Infectious Diseases Report, www.cdc.gov/ncidod/diseases/hepatitis/c, 2001), and infections can be divided into three groups: resolved infections (15%), such as in patients that are anti-HCV antibody positive and HCV RNA negative and have normal alanine aminotransferase (ALT) levels; persistent infections/death (15%), as in patients that are HCV antibody positive and HCV RNA positive and have elevated ALT levels; and benign persistent disease (70%), as in patients that are HCV antibody positive and HCV RNA positive and have normal or mildly elevated ALT levels.
Data from the Retrovirus Epidemiology Donor Study and recent data from the American Red Cross show that the frequency of HCV-positive U.S. blood donations is 1.12 per 1,000. The American Red Cross rate for first-time donors is 4.25 per 1,000, and for repeat donors it is 0.03 per 1,000. The window of infectivity before seroconversion is calculated to be approximately 70 days (8). With the licensed serological assays, it is estimated that up to 10 HCV infectious units per million donations escape detection and enter the blood supply (8, 15).
Detection of HCV, like that of HIV-1, is also based on serologic screening for antiviral antibodies, using enzyme-linked immunosorbent assays and confirmation with a recombinant strip immunoblot assay. Even though implementation of these tests has significantly reduced the prevalence of posttransfusion HCV infection in the United States and tests with improved sensitivity have allowed earlier detection, the risk of contracting HCV through transfusion still exists. HCV immunoserologic tests are limited by the prolonged, 70-day window of seronegativity after acute infection, the inability to differentiate between active and resolved infections, and false-negative results in rare patients with chronic, antibody-negative infection. Antigen detection tests for HCV are not yet available for blood bank screening. Use of NAT allows the detection of HCV infection about 50 to 60 days earlier than current antibody-based tests, resulting in a window period of approximately 11 days (4, 13, 14, 15, 23, 41).
Detection of viral nucleic acids as markers for HIV-1 and HCV infection and disease progression has become widely accepted as a research tool (3, 7, 34, 37). Nucleic acid tests allow earlier detection of infection because they directly test for the presence of circulating virus rather than measure the individual's antibody response to the virus. In addition, NAT exhibits a very high sensitivity of detection, serves as an independent method for confirming infection in indeterminate serology cases, helps to discriminate between chronic and resolved HCV cases, and confirms infections in HIV- and HCV-seropositive newborns. Several technologies for the detection of HIV and/or HCV nucleic acids are currently available, such as branched DNA signal amplification (10, 12, 18, 39), nucleic acid sequence-based amplification (40, 42), strand displacement amplification (29, 44), the ligase chain reaction (24, 27), transcription-mediated amplification (21, 28), and PCR (1, 25, 31, 38). The main shortcomings of the first-generation NAT technologies, rendering them unsuitable for high-throughput screening, include technical difficulty, significant hands-on time, lengthy incubation times, large sample volume requirements, high cost, relatively high rate of false positives, and inconsistency in detection of genetic variants (11, 17, 33, 34).
To provide a method suitable for high-throughput screening, semiautomated assays were developed for codetection or individual detection of HIV-1 and HCV RNAs. The HIV-1/HCV assay and the corresponding HIV-1 and HCV discriminatory assays utilize three proprietary technologies: (i) target capture-based sample preparation, (ii) transcription-mediated amplification, and (iii) hybridization protection assay. All three assays incorporate an internal control to validate each reaction. A positive result in the HIV-1/HCV multiplex assay does not distinguish between the presence of HIV-1 and HCV in the specimen; therefore, individual HIV-1 and HCV discriminatory assays are subsequently used to identify the virus present. HIV-1 and HCV discriminatory assays could also be used as highly sensitive qualitative assays for diagnostic and epidemiological applications. The assays detect all known HIV-1 and/or HCV subtypes and have sensitivities designed to reduce the window period between infection and detection while maintaining the specificity to discriminate between positive and negative specimens. These assays, described in detail below, are currently used in several blood donation locations under investigational new drug applications and have received a Biologics license from the U.S. Food and Drug Administration.
| MATERIALS AND METHODS |
|---|
|
|
|---|
The 3' dA sequences of the capture oligonucleotides hybridize to dT oligonucleotides that are covalently bound to the magnetic microparticle solid phase. The purpose of the magnetic particles in sample preparation is to specifically capture and concentrate the HIV-1 and HCV RNA targets from the sample. The superparamagnetic particles are approximately 1 µm in diameter, with a hydrophilic surface that is derivatized with oligo(dT). Under capture conditions, the dT tail binds to the capture oligonucleotides. Target viral RNAs are captured onto magnetic microparticles, readily separated from other plasma components in a magnetic field, and washed twice.
Amplification with TMA. The Gen-Probe Transcription-Mediated Amplification (TMA) (21, 28) technology exponentially amplifies the captured HIV-1 and HCV viral RNAs. TMA is an isothermal transcription-based amplification technique that utilizes a two-enzyme system to produce RNA amplicon via DNA intermediates. TMA requires bacteriophage T7 RNA polymerase, Moloney murine leukemia virus reverse transcriptase (RT), deoxyribo- and ribonucleoside triphosphates, T7 promoter-primers and non-T7 primers, and target nucleic acid RNA. Reverse transcriptase is a recombinant version of the Moloney murine leukemia virus reverse transcriptase that is expressed in Escherichia coli. Moloney murine leukemia virus reverse transcriptase has both RNA-dependent and DNA-dependent DNA polymerase activities as well as RNase H activity. T7 RNA polymerase is a recombinant version of the bacteriophage T7 RNA polymerase that is also expressed in E. coli. There are five T7 promoter-primers and five non-T7 primers used in the assay. Four of the primers are involved in the amplification of HCV sequences, four additional primers are involved in the amplification of HIV polymerase and internal control sequences, and the remaining two are involved in the amplification of HIV LTR sequences. The steps of TMA are outlined in Fig. 1.
|
The Dual Kinetic Assay utilizes the kinetic discrimination of chemiluminescent emissions. A probe with rapid emission kinetics, known as a flasher (ortho-fluoroacridinium ester), is used to detect the internal control. Probes with slower emission kinetics, termed glowers (2-methylacridinium ester), are used to detect target viral nucleic acids. At a given pH and temperature, the rate of chemiluminescent emissions from a flasher is more rapid than that from a glower. There are six detection probes used in the assay. Two glower probes are directed against HCV, one glower probe is directed against the LTR region of HIV, two glower probes are directed against the HIV polymerase region, and a flasher probe detects the internal control sequence.
The exponential tail fit algorithm (32) is used to deconvolute the two simultaneous signals from every reaction mixture containing both internal control and target RNA. The shape of the glower signal over time is scalable, and its geometry is constant under any given set of pH and temperature conditions, regardless of the magnitude of the signal. As such, the glower signal at any given time is directly proportional to the total integrated signal/time function. By design, the flasher signal from the hybridized internal control probe is brief and contributes a negligible amount to the magnitude of the glower signal at later time points.
HIV-1 and HCV discriminatory assays. The discriminatory assays utilize the same three main steps as the multiplex assay (target capture, TMA, and HPA), and the same basic assay procedure is followed. The discriminatory assays are based on chemistry that is identical to the HIV-1/HCV assay except that the labeled probes used and, in the case of the HCV discriminatory probe reagent, the HCV-specific labeled oligonucleotides are present at a higher concentration.
Assay run, calibrators, and controls. Each TMA assay run can accommodate up to 100 reactions, including nine assay calibrators and up to 91 specimens, and a single operator can manage two assay runs simultaneously. The results obtained from the nine calibrators contained in each TMA assay run are used to determine the validity of the run and to establish the assay cutoffs as described below. The negative calibrator is defibrinated normal human plasma, which is nonreactive for hepatitis B virus surface antigen, antibodies to HIV-1 and HIV-2, and antibodies to human HCV when tested by Food and Drug Administration (FDA)-licensed assays. The HIV-1 positive calibrator consists of heat-inactivated, HIV-1-positive plasma in defibrinated normal human plasma that is nonreactive for hepatitis B virus surface antigen and anti-HCV when tested by FDA-licensed assays. The heat-inactivated HIV-1 virus is present at 500 ± 200 copies/ml. Similarly, the HCV positive calibrator consists of heat-inactivated HCV-positive plasma in defibrinated normal human plasma that is nonreactive for hepatitis B virus surface antigen, anti-HIV-1, and anti-HIV-2 when tested by FDA-licensed assays. The heat-inactivated HCV is present at 560 ± 260 copies/ml.
The internal control serves as a general monitor of assay performance for each specimen tested, as it is added to each test specimen and assay calibrator reaction. The internal control consists of an in vitro-synthesized transcript containing a portion of HIV-1 and a unique sequence targeted by the internal control probe. The internal control signal is distinguished from HIV-1 and HCV analyte signal by the differential kinetics of light emission of the labeled probes used for each target. Prior to carrying out target capture, the internal control is added to a newly opened target capture reagent bottle to
250 copies/reaction. As it is added directly to the target capture reagent, the internal control controls for false-negative results that could occur due to problems in target capture, amplification, or detection.
Formula-derived assay cutoff values. Two cutoffs are determined for the TMA assays, termed the analyte cutoff and internal control cutoff, for the analyte and internal control signals, respectively. These cutoff values are derived from the analyte and internal control RLU signals of the nine assay calibrators that are run at the beginning of each assay run.
Formulas were established to allow determination of a floating analyte cutoff in each assay that falls within the range of RLU values indicated by receiver operating characteristic (ROC) analysis to achieve
99.5% sensitivity and specificity. By using a floating cutoff rather than a set value, day-to-day variations due to reagents, operators, and instruments are taken into account. The formulas add a portion of the HIV-1 positive calibrator analyte signal and/or the HCV positive calibrator analyte signal to the negative calibrator analyte signal as defined in Table 1 for the three assays. When a sample demonstrates an analyte signal/analyte cutoff (S/CO) ratio of
1, it is considered reactive for the target RNA.
|
1, it is considered reactive, and an internal control S/CO ratio of <1.0 indicates a nonreactive result. In negative samples, a positive internal control result validates the reaction. When samples are reactive for analyte signal as determined from the analyte S/CO, the internal control result is not used to validate the reaction. Instrumentation. The testing system is composed of the following components: TECAN GENESIS series RSP model 150/8 Instrument pipettor, Gen-Probe target capture system revision B, Gen-Probe LEADER HC+ luminometer, and dedicated personal computer with TMA blood testing data reduction software (Chiron Procleix 2.0), circulating waterbaths for incubation steps, and multitube vortexers for mixing steps. Data reported here were transferred from the LEADER HC+ instruments to a structured query language server database, from which the data were retrieved and analyzed with Microsoft Excel 97.
Specificity testing. Specificity data were generated by testing frozen EDTA plasma samples from individual normal donors obtained from Gary Tegtmeier of the Community Blood Center of Greater Kansas City (Kansas City, Mo.). Approximately 1,500 samples were tested with three lots of reagents.
Analytical sensitivity evaluation. Analytical sensitivity panels comprised of HIV-1 type B and HCV subtype 1a at concentrations of 300, 100, 30, 10, 3, 1, and 0 viral copies/ml were used to evaluate assay sensitivity. The HIV-1 panel members were prepared by serial dilution of an HIV-1 type B isolate from a tissue culture supernatant. HCV panel members were made by serial dilution of patient plasma samples containing HCV subtype 1a. The viral stocks used to make the panel were quantified using in-house HIV-1 or HCV quantitative assays that were calibrated according to the Virology Quality Assurance laboratory of the AIDS Clinical Trials Group or World Health Organization (WHO) standards, respectively. The analytical sensitivity panel members were frozen in 5.6-ml single-use aliquots and thawed at room temperature immediately before they were tested in the TMA assays. Approximately 80 replicates of each were tested with each of three reagent lots in each of the three test assays. To prepare the coinfected panels used in this study, 33 ml of HIV-1-infected plasma and 33 ml of HCV-infected plasma were combined at the appropriate concentration. Limits of detection of HIV-1 and HCV were also determined using NAT standards obtained from the Center for Biologics Evaluation and Research (CBER, FDA), the WHO, and Bioclinical Partners, Inc. (Franklin, Mass.).
Effect of donor and donation factors on sensitivity and specificity. The specificity and sensitivity of the TMA HIV-1/HCV assay, the HIV-1 discriminatory assay, and the HCV discriminatory assay were evaluated with respect to the effects of donors and donation factors in various types of specimens. Conditions, factors, and sample characteristics that were evaluated included the following: (i) infections other than HIV-1 or HCV (herpes simplex virus types 1 and 2, human T-cell lymphotrophic virus types I [n = 8] and II [n = 8], hepatitis A virus, hepatitis B virus, HIV-2, cytomegalovirus, Epstein-Barr virus, rubella virus, and parvovirus B19) and vaccination status (influenza and hepatitis B virus [n = 10 each]) (ProMedDx, LLC, Norton, Mass.; BioClinical Partners, Inc., Franklin, Mass.; and Boston Biomedica, Inc., West Bridgewater, Mass.); (ii) specimens contaminated with bacterial, yeast, or fungal pathogens (Staphylococcus epidermidis, Staphylococcus aureus, Micrococcus luteus, Corynebacterium diphtheriae, Propionibacterium acnes, Candida albicans, and Pneumocystis carinii) (normal blood donations from the Community Blood Center of Greater Kansas City spiked to 105 CFU/ml); (iii) plasma samples from donors with autoimmune and other diseases (rheumatoid arthritis, rheumatoid factor, antinuclear antibody, lupus, multiple sclerosis, multiple myeloma, hypergammaglobulinemia [immunoglobulin G {IgG} and IgM], alcoholic cirrhosis, and elevated alanine aminotransferase levels) (ProMedDx, LLC; BioClinical Partners, Inc.); (iv) specimens containing potentially interfering substances (hemolyzed, icteric, and lipemic specimens) (ProMedDx, LLC) and an analytical panel prepared in-house containing bilirubin (200 mg/liter), lipids (2,752 mg/dl), hemoglobin (5,000 mg/liter), or protein (225 g/liter); (v) serum samples and plasma specimens collected in various anticoagulants (acid citrate dextrose, plasma preparation tube with dipotassium EDTA as the anticoagulant, dipotassium EDTA and tripotassium EDTA, sodium citrate, and sodium heparin) (ProMedDx, LLC).
Overall, approximately 830 HIV-1 and HCV antibody-negative specimens were tested in the three assays for specificity analysis. Aliquots of each specimen were also spiked with HIV-1 or HCV target (200 copies/ml) for analysis of assay sensitivity. For comparison to the assorted conditions investigated, control specimens from healthy blood donors were used as HIV-1 and HCV antibody-negative specimens for specificity analysis, and healthy donor specimens were spiked with HIV-1 or HCV for sensitivity analysis (Community Blood Center of Greater Kansas City, Kansas City, Mo.).
Assay reproducibility. The reproducibility of the HIV-1/HCV assay was evaluated with respect to variations introduced by operator, reagent lot, LEADER HC+ luminometer system, and assay run. Three operators carried out the assay using three luminometers and three lots of reagents. Fifty-four runs were completed for the study and included in the analysis. The HIV-1 and HCV samples evaluated were panels containing HIV-1 type B, HIV-1 group O, or HCV 1a at 100 copies/ml or HCV 2b at 300 copies/ml. The HIV-1-containing panels were constructed by dilution of cultured virus in HIV-1/HCV-negative serum, and gentamicin sulfate was included at 12.5 µg/ml. Similarly, the HCV-containing panels were made by diluting HIV-1-negative serum containing either HCV genotype 1a or HCV genotype 2b into HIV-1/HCV-negative serum, with gentamicin sulfate included at 12.5 µg/ml.
Statistical determination of assay cutoff values. ROC curves were used to assess the diagnostic performance of the assay in terms of specificity and sensitivity for each possible cutoff value of the test. Briefly, the sensitivity and specificity data were combined and sorted according to ascending RLU result. With each specific RLU result, or k, the number of reactive and nonreactive decisions that would occur with a cutoff >k but less than the following data point was calculated. The sensitivity (number of positive decisions/true-positive samples) and specificity (number of negative decisions/true-negative samples) for the cutoff point was determined for each cutoff point implemented. The use of actual data points to determine the cutoff values used, rather than using constant small intervals, or bins, results in variable bin width but provides a more optimal resolution of the acceptable cutoff values. These calculations were carried out using a Visual Basic program in Microsoft Excel 97, and the program was verified through comparison to results obtained with SAS software (SAS Institute, Cary, N.C.).
The parameters of sensitivity and 1 - specificity were then plotted for each cutoff value as an ROC curve, where the upper left-hand corner of the curve identifies the specificity and sensitivity limits of the assay. From these analyses, a range of cutoff values was determined that allows
99.5% sensitivity and
99.5% specificity in accordance with the assay requirements. The point at which the sensitivity and the specificity were mutually maximized was also determined from the ROC curve analysis.
Samples containing at least 100 copies of HIV-1 subtype B/ml or 300 copies of other HIV-1 subtypes/ml were included in the ROC analysis of the HIV-1/HCV assay and the HIV-1 discriminatory assay. Samples containing at least 100 copies of HCV subtype 1a/ml or 300 copies of other HIV-1 subtypes/ml were included in the ROC analysis of the HIV-1/HCV assay and the HCV discriminatory assay. Panel samples containing 0 copies/ml and the specimens from healthy blood donors were included in the ROC analyses as specificity samples. The specificity and sensitivity samples tested under various donor and donation factor conditions were also included in the ROC analyses as they met these described qualifications as well.
| RESULTS |
|---|
|
|
|---|
|
95% detection at 100 copies/ml for HIV-1 type B or HCV type 1 was shown. Detection was seen down to 2.7 copies/ml for HIV-1 (Bioclinical Partners HIV-1 Reference Panel) and to 0.1 IU/ml for HCV type 1 (WHO HCV 96/790). Detection of HIV-1 genotype samples (CBER HIV-1 Genotype Panel) was also demonstrated at concentrations below 100 copies/ml (data not shown) (35; P. N. Lelie et al., unpublished data). Assay sensitivity under coinfection conditions. The analytical sensitivities of the HIV-1/HCV assay as well as the HIV-1 discriminatory assay and HCV discriminatory assay were evaluated under conditions of HIV-1 and HCV coinfection. The coinfection panels (Table 3) essentially consisted of the following combinations: one virus present at 107 copies/ml and the other at 30 copies/ml (panels A and B), and one virus at 30 copies/ml and the other virus at 0 copies/ml (panels C and D). Upon testing of 10 replicates of panels A to D in the HIV-1/HCV assay, 100% positivity was achieved regardless of the combination of viruses included (Table 3). The signal-to-cutoff ratios observed in the multiplex assay ranged from 9.66 for panel C, containing only HCV at 30 copies/ml, to 34.12 for panel A, containing both viruses. All panel members containing HIV-1 were 100% reactive in the HIV-1 discriminatory assay, including panel B, which contained 107 copies of HCV/ml and only 30 copies of HIV-1/ml.
|
Assay specificity in healthy blood donor specimens.
In nonreactive samples, the internal control S/CO ratio indicates whether the reaction was complete in each individual reaction by controlling for the steps of target capture, amplification, and detection. If the internal control S/CO ratio is
1.0, the reaction is considered valid, while any result less than 1.0 is interpreted as an internal control failure or invalid result. However, if the analyte result is positive, the result is treated as valid regardless of the internal control signal. In testing the 1,500 healthy blood donor specimens, there were incidences of initial internal control failure in each of the TMA assays (Table 4), which generally indicates insufficient amplification due to nonspecific inhibition or other assay protocol deficiency in one or more of the steps of target capture, amplification, or detection. In all cases, retesting of the samples with initially invalid internal control signals resulted in valid internal control amplification, indicating that the sample itself did not inhibit the assay reaction.
|
1.0 are considered reactive, or positive for HIV-1 and/or HCV RNA, while those with an analyte S/CO ratio of <1.0 are considered nonreactive, or negative for HIV-1 and HCV RNAs. Of 1,500 specimens from donors with negative serology tested in the HIV-1/HCV assay, only one was initially reactive. This sample was not repeatedly reactive upon retesting (Table 4). The initial reactive result could be the result of amplicon or target contamination of the sample or incomplete selection during the HPA step. When these specimens were tested in the discriminatory assays, there were no initial reactive samples in the HIV-1 discriminatory assay, and only 2 of the 1,500 normal specimens tested in the HCV discriminatory assay were initially reactive with nonreactive results upon retest (Table 4). The distribution of internal control S/CO ratios in the HIV-1/HCV assay is summarized graphically in Fig. 2A; the average internal control S/CO value was 2.04 with a standard deviation (SD) of 0.13. This demonstrates clear separation of 7.6 SD from the cutoff of 1.0. The distribution of internal control S/CO ratios for the discriminatory assays is similar to that shown in Fig. 2A for the HIV-1/HCV assay, with mean ratios ± SD of 2.03 ± 0.14 and 2.03 ± 0.15 for the HIV-1 and HCV discriminatory assays, respectively. Likewise, the distribution of analyte S/CO ratios for the negative specimens is also summarized graphically in Fig. 2A. The mean analyte S/CO value of this population was 0.20 with an SD of 0.06 and demonstrates a clear separation of 13.0 standard deviations from the cutoff of 1.0. The distribution of analyte S/CO ratios for the discriminatory assays is similar to that shown in Fig. 2A for the HIV-1/HCV assay, with mean S/CO ratios ± SD of 0.17 ± 0.05 and 0.12 ± 0.07 for the HIV-1 and HCV discriminatory assays, respectively.
|
Donor and donation effects on assay sensitivity and specificity.
The specificities and sensitivities of the HIV-1/HCV assay, the HIV-1 discriminatory assay, and the HCV discriminatory assay were evaluated with two reagent lots with respect to the effect of donor and donation factors. Conditions and characteristics evaluated included infections other than HIV-1 and HCV, specimens contaminated by bacterial, yeast, or fungal pathogens, problematic plasma samples from donors with autoimmune and other diseases, specimens containing potentially interfering substances (hemolyzed, icteric, and lipemic specimens and samples containing high levels of bilirubin, lipids, albumin, and hemoglobin), and plasma specimens collected in various anticoagulants and serum specimens. Approximately 10 individual specimens for each category were tested, for a total of
830 specimens. These specimens, which were HIV-1 and HCV antibody negative, were tested to evaluate assay specificity.
Aliquots of each specimen were also spiked with approximately 200 copies of HIV-1 or HCV/ml, and assay sensitivity was assessed. Control specimens for these studies consisted of 91 samples from healthy blood donors for specificity evaluation. Additional specimens from healthy blood donors were also spiked with HIV-1-infected plasma (targeted input of 200 copies/ml, n = 91) or HCV-infected plasma (targeted input of 200 copies/ml, n = 91) for assessment of assay sensitivity.
Table 5 shows the positivity rates obtained with samples representing various donor and donation factors tested in the HIV-1/HCV assay. All HIV-1- and HCV-seronegative specimens, regardless of specimen type, displayed positivity rates of 0%, as expected for antibody-negative samples. Furthermore, the positivity rate of 0% and confidence intervals of 0 to 0.4% were not statistically different from those determined with the control specimens (0% [CI = 0 to 4%]). Similar results were obtained in the HIV-1 and HCV discriminatory assays, with 0% reactivity in both assays and confidence intervals of 0 to 0.4% and 0 to 0.5%, respectively (data not shown). There was no particular group of donors or type of donation factor that resulted in nonspecific reactivity or false-positive results.
|
Overall, the specificity and sensitivity of the three assays demonstrated less than 0.5% repetitive reactive rates with negative specimens and greater than 95% detection at higher than 100 copies of HIV-1 or HCV/ml under all conditions tested.
Reproducibility of the HIV-1/HCV assay. A total of 2,151 valid reaction results were analyzed for assay reproducibility in the HIV-1/HCV assay to determine the influences of operator, reagent lot, luminometer, and run on assay variability. Table 6 shows the positivity rates and coefficients of variance (CV) obtained with in-house HIV-1- and HCV-infected panels. Overall sensitivity results for this study indicated that the HIV-1 type B and HCV 1a panels containing 100 copies/ml both achieved high (99.6% [CI = 98.7 to 100%]) detection rates. Testing of the HIV-1 group O panel containing 100 copies/ml and the HCV 2b panel containing 300 copies/ml resulted in 97.4% (CI = 95.7 to 98.6%) and 95% (CI = 92.8 to 96.7%) positivity rates, respectively.
|
Statistical analysis of assay specificity and sensitivity.
The specificity specimens and sensitivity samples containing
100 copies of HIV-1, HCV, or both per ml were statistically analyzed by ROC as described in Materials and Methods. Conventional ROC plots were constructed for data obtained with the HIV-1/HCV assay, the HIV-1 discriminatory assay, and the HCV discriminatory assay, as demonstrated in Fig. 3. There are two plots presented for the HIV-1/HCV assay, with a separate ROC analysis for each of the targets, HIV-1 and HCV. This analysis clearly shows the very high specificity and sensitivity of the three assays. In all cases, cutoff ranges, which allow for
99.5% sensitivity and
99.5% specificity, were determined from the analyses (Table 7). Results obtained from the ROC analysis of analyte RLUs of the HIV-1/HCV assay indicated that cutoff values between 37,777 and 284,560 RLU achieve greater than 99.5% specificity and sensitivity for the detection of HIV-1. Likewise, cutoff values between 37,777 and 121,856 RLU achieve the same specificity and sensitivity in the HIV-1/HCV assay for the detection of HCV. Analysis of the HIV-1 discriminatory assay determined a cutoff range of 29,026 to 152,941 RLU, and the cutoff range for the HCV discriminatory assay was 48,860 to 132,007 RLU. The cutoff values that maximized the sensitivity and specificity were quite similar between assays, ranging from 38,198 to 62,880 RLU (Table 7).
|
|
99.5% sensitivity and specificity. The cutoff formula for each assay averages the analyte signal from the negative calibrators and then, depending on the assay, adds a fraction of the analyte signal values obtained for the HIV-1 and/or HCV positive calibrators (Table 1).
Table 8 demonstrates the calculated cutoff determined from the 245 runs of the HIV-1/HCV assay that were analyzed by ROC in Fig. 3. An average calculated cutoff of 51,076 RLU was indicated, falling within the ranges of 37,777 to 284,560 RLU and 37,777 to 121,856 RLU determined by ROC analysis to achieve
99.5% sensitivity and specificity in the detection of HIV-1 and HCV, respectively (Table 7). The highest and lowest analyte cutoff calculated for these 245 runs also fell within the ranges defined in Table 7. A total of 133 runs were analyzed in the HIV-1 discriminatory assay and were found to have an average calculated cutoff of 50,139 RLU. This cutoff falls within the range of 29,026 to 152,941 RLU defined for the same runs by ROC analysis to achieve
99.5% sensitivity and specificity. Even the highest and lowest calculated cutoffs within these runs (Table 8) fall within the defined range.
|
| DISCUSSION |
|---|
|
|
|---|
99.5% for samples containing HIV-1 or HCV RNA at greater than or equal to 100 copies/ml. This was achieved with specimens of varied characteristics and was confirmed under conditions of coinfection of the viruses. The sensitivity of the assay is much greater than the established assay goal of 100 copies/ml, as probit analysis predicts 95% detection rates for HIV-1 type B and HCV subtype 1a at 13 and 34 copies/ml, respectively (Table 2). Results from testing NAT standards corroborates these results (P. N. Lelie, A. A. van Drimmelen, H. T. M. Cuypers, S. Best, E. M. Dax, S. L. Stramer, C. Hyland, J. P. Allain, P. Moncharmont, C. Defer, M. Nubling, A. Glauser, M. da Silva Cardoso, J. F. Viret, M. Lankinen, L. Grillner, U. Wirthmuller, J. Coste, V. Schottstedt, and B. Masecar, unpublished data). These limits of detection are lower than those reported previously for HIV-1 or HCV assays using reverse transcription-PCR or branched DNA technology (
50 to 100 copies/ml) (10, 12). Furthermore, the assay detects the most divergent HIV-1 and HCV subtypes, including HIV-1 group O and HCV 2b, with similar sensitivities (Table 6) (16, 26).
The statistical relationship of assay specificity and sensitivity obtained by ROC plots (Fig. 3; Table 7) demonstrated that all three assays show excellent discrimination between positive and negative samples and a broad cutoff range within which the specificity and sensitivity are greater than or equal to 99.5%, consistent with the proposed claims of the assay. The wide separation of average S/CO and the high degree of sensitivity and specificity of the assay will help ensure accurate detection of HIV-1 or HCV in the blood bank setting or in other diagnostic applications. Furthermore, the use of floating analyte cutoffs reduces the effects of day-to-day variations and allows the calculation of cutoff values which consistently fall within a range of RLU analyte cutoff values defined by ROC analysis which allows the assay to achieve
99.5% specificity and sensitivity at 100 copies/ml. Overall, the ROC analysis indicates that in the hands of various users, the assay will consistently determine an analyte cutoff value that allows reproducible sensitivity and specificity of detection of HIV-1 and/or HCV RNA.
The overall specificity and sensitivity goals of 100 copies/ml and
99.5% were reached even when potentially interfering donor factors were present in specimens, such as additional viruses, pathogens, or potentially interfering substances, or when samples from patients with autoimmune or other diseases were tested (Table 5). This robustness of the HIV-1/HCV assay, even when testing nontypical specimens, was observed for all three assays. The nature of the target capture technology minimizes interferences, as potentially inhibitory substances are removed in the first step of the assay. In addition, donation factors such as the chosen anticoagulant for sample collection had no effect on the specificity or sensitivity of the assays. This is in contrast to some reports of target recovery issues with certain anticoagulants with other technologies (20) and implies that the TMA assays could readily be applied to additional targets and specimen types in a variety of applications.
The signals of the internal control and calibrators have been optimized to result in clear separation of the internal control from the cutoff (
8 SD) and distinct analyte S/CO ranges for the three calibrators (Fig. 2). Similar to the information provided by calibrator S/CO distribution, the analyte S/CO values in the HIV-1/HCV assay allow preliminary discrimination between HIV-1 signal and HCV signal (Table 3). Figure 3D, containing HIV-1 RNA only, has an S/CO ratio of 18.46, similar to that of the HIV-1 positive calibrator. Likewise, the S/CO ratios for Fig. 3C, containing HCV RNA only, is 9.66 and essentially the same as that of the HCV positive calibrator signal. In addition, the panels containing both HIV-1 and HCV show the general trend of having an S/CO ratio that is additive to that which is observed for the individual targets. While this does not eliminate the need for using the discriminatory assays to determine whether the sample is infected with HIV-1 or HCV or both, it does allow the user to have some indication of the reactive target.
The multiplex format of the HIV-1/HCV assay has advantages with respect to time and cost compared to assays designed with a single-target format. The time needed to complete initial screening is cut in half due to the ability to amplify multiple targets simultaneously. Also, the batch sample processing method of this assay is highly effective and allows the processing of 200 specimens (either single donor unit or plasma pool sample) and calibrators by a single operator in less than 2 h and completion of 182 test results for each target in 6 h. This is in contrast to another recently presented methodology that describes assay batch sizes that are approximately 10 times smaller (45).
The necessity for NAT of blood donations in addition to or in lieu of some of the currently licensed serological techniques is clear due to the ability of these assays to significantly reduce the window period between infection and detection (3, 15). A negative result by serological screening may be a false-negative result if the donor was recently infected. Due to the additional sensitivity and specificity provided by NAT, genomic detection could readily replace screening for HIV-1 p24 antigen, which has not yielded more than six instances of antigen-positive, antibody-negative donations in over 24 million specimens tested (8, 37). The added sensitivity of NAT would likely have detected these six infected donations and may have identified additional infected specimens. The technology of the HIV-1/HCV assay could also be used instead of techniques such as the Western blot and recombinant strip immunoblot assay, assays currently implemented for confirmation of HIV-1 and HCV antibody-positive specimens. Detailed studies with commercially available seroconversion panels further demonstrate the utility of implementing NAT to replace these HIV-1 and HCV confirmation techniques (22, 43).
The test system discussed in this report is currently used in various blood donation locations under investigational new drug applications approved by the FDA. As of August 2000, approximately 11 million blood donor units had been tested using the HIV-1/HCV and discriminatory assays at the American Red Cross, America's blood centers, and America's independent blood centers. Within this study set, 43 HCV NAT-reactive, seronegative donations and three HIV-1 NAT-reactive, p24 antigen-negative and seronegative donations were identified (37).
In addition to earlier detection of HIV-1 and HCV in blood donations and prevention of infection by transfusion, the use of the HIV-1/HCV assay technology in diagnostic applications would provide increased sensitivity for monitoring viral RNA status. Implementation of the HIV-1 and HCV discriminatory assay technologies would improve the monitoring of patients undergoing treatment by providing lower detection limits than the ones provided by current quantitative assays, aiding in the evaluation of therapeutic endpoints. The ability to detect multiple subtypes of HIV-1 and HCV further supports the worldwide utility of the TMA assays. Furthermore, the technology is readily adaptable to include other targets such as hepatitis A virus, hepatitis B virus, and parvovirus as the need for blood bank screening and diagnostic monitoring of these and/or additional infectious agents evolves.
| ACKNOWLEDGMENTS |
|---|
This project has been funded in part with federal funds from the National Heart, Lung and Blood Institute under contract no. NHLBI-HB-67130.
| FOOTNOTES |
|---|
| REFERENCES |
|---|
|
|
|---|
This article has been cited by other articles:
| ||||||||||||||||||||