Previous Article | Next Article ![]()
Journal of Clinical Microbiology, October 2002, p. 3558-3564, Vol. 40, No. 10
0095-1137/02/$04.00+0 DOI: 10.1128/JCM.40.10.3558-3564.2002
Copyright © 2002, American Society for Microbiology. All Rights Reserved.
Indiana University School of Medicine, Indianapolis, Indiana,1 Digene Corporation, Gaithersburg, Maryland2
Received 4 February 2002/ Returned for modification 11 April 2002/ Accepted 2 July 2002
| ABSTRACT |
|---|
|
|
|---|
= 0.97) and 99.8% (
= 0.97), respectively. When the HC2 assay was performed with a semiautomated system application designed for high throughput, it demonstrated high sensitivity and a high specificity for detection of both Chlamydia trachomatis and Neisseria gonorrhoeae. | INTRODUCTION |
|---|
|
|
|---|
The Centers for Disease Control and Prevention has recommended that all sexually active women under the age of 26 be screened for C. trachomatis infection (9), and the European guidelines for the management of chlamydial and gonococcal infections recommend similar targeted screening for both of these organisms (4, 27). Beginning in the year 2000, the Health Employer Data Information Set, a system used nationwide to assess the performance of managed care health services in the United States, adopted this screening recommendation as a criterion measure for evaluation of the performance of insurance programs (19). As a result of these recommendations and of studies demonstrating the cost-effectiveness of chlamydial screening programs (19, 21), increases in the number of patients screened have been seen throughout the country (5, 23).
Universal screening practices are now more feasible since highly sensitive commercially available nucleic acid amplification tests allow transport of specimens from clinical settings to remote laboratories. Although these assays have demonstrated excellent sensitivities and specificities in clinical trials (6, 28, 29, 30), they require extreme technical precision in order to achieve maximum performance (18, 22, 28). With the assays available at present, this translates into labor-intensive diagnostic methods, often with low-throughput capacity (2, 3, 16). In this study we evaluated a semiautomated system application of the signal amplification-based Hybrid Capture 2 CT/GC (HC2) assay that allows testing of 352 samples for both C. trachomatis and N. gonorrhoeae during an 8-h shift.
The Rapid Capture System (RCS) is a programmable 96-well microplate processor that integrates liquid handling, plate handling, incubation, shaking, and washing via software specifically designed to run the HC2 assay (Digene Corporation, Gaithersburg, Md.). We evaluated the performance of the HC2 assay with the RCS application (HC2-RCS) using prospectively collected endocervical samples tested by the HC2 assay performed manually (HC2-M), the COBAS Amplicor (Roche Diagnostic Corporation, Indianapolis, Ind.) CT/NG assay (PCR), and culture. Here we present the first evaluation of the performance characteristics of HC2-RCS for detection of C. trachomatis and N. gonorrhoeae.
| MATERIALS AND METHODS |
|---|
|
|
|---|
The second population consisted of consenting women attending an obstetrics and gynecology clinic in Porto Alegre, Brazil, who were enrolled in a study to evaluate the Digene Hybrid Capture II HPV DNA assay. Diagnostic testing of a separate specimen for STDs was included in the design of the original study for which these samples were collected. The results of tests performed at Digene for human papillomavirus and C. trachomatis-N. gonorrhoeae were reported to the clinic in Brazil if the specimen had been kept under stable conditions for testing (at ambient temperature for up to 2 weeks and at -20°C for up to 3 months). Archived specimens from Brazil tested at Indiana University were stored at ambient temperature for 5 to 8 days, shipped at ambient temperature to the United States, and subsequently stored at -20°C for 3 to 6 months. The only identifiers on these samples were specimen numbers, and testing at Indiana University was approved by the Indiana University Institutional Review Board.
HC2-RCS and HC2-M. The HC2 assay is performed as a combined test for C. trachomatis and N. gonorrhoeae (CT/GC assay), followed by identification of the infecting pathogen: C. trachomatis (CT-ID assay) and N. gonorrhoeae (GC-ID assay). The HC2 assays detect DNA-RNA hybrids with signal amplification technology. Therefore, these assays are not affected by DNA amplification inhibitors and do not include a DNA amplification control. The CT/GC, CT-ID, and GC-ID assays follow a single testing protocol with assay-specific reagents. The initial denaturation step is performed manually for both HC2-RCS and HC2-M. Samples were processed in batches of up to 96 by addition of a denaturation reagent and incubation at 65°C for 45 min. After denaturation, samples were either tested immediately or held at -20°C for subsequent testing.
HC2-M was performed as described previously (10) with endocervical samples, the only specimen type approved in the United States for use in the HC2 assay for testing for C. trachomatis and N. gonorrhoeae. For HC2-RCS, denatured samples were loaded onto the RCS, in which all assay steps prior to signal detection were performed automatically by the RCS application. Each run of the RCS has the capacity to test up to four 96-well plates, consisting of 352 samples (88 per plate) and 32 controls (8 per plate), but has a flexible format that allows any number of specimens from 1 to 352 to be run. To maximize the work flow with the RCS application, testing was done with batches of 352 samples whenever possible. The total run time for the HC2 assay with the RCS application is approximately 7.5 h, with 3.5 h of uninterrupted hands-free time.
Test results were calculated by using assay-specific software accompanying the DML 2000 luminometer. Relative light units (RLUs) for the positive and negative controls are used to calculate the run-specific cutoff (CO), and specimen results are reported as RLU/CO ratios. In clinical testing settings, samples that yield an RLU/CO value of
1.0 by the CT/GC assay would routinely be retested by the CT-ID and GC-ID assays to identify the infecting organism. In these studies, both the CT-ID and GC-ID assays were performed regardless of the CT/GC assay result, allowing interpretation of results by one of the two methods described below. RLU/CO values between 1.0 and 2.5 by the CT-ID or GC-ID assay were considered equivocal, and the samples were retested in duplicate. These assay results were interpreted by using an equivocal-zone retesting algorithm and were considered positive if the results of at least two of the three tests (the initial test and the two retests) had RLU/CO values
1.0. Denatured samples in STM contain sufficient volume for 15 to 18 assays, which allows retesting to be performed, if necessary.
The performance of HC2-RCS was evaluated by using two separate analyses with different interpretive criteria based on the results of the different tests. The first analysis used results based on the outcome of the CT/GC assay followed by the identification assays (CT/GC algorithm), while the second analysis was based on the results of the individual identification assays, including those obtained with the equivocal-zone retesting algorithm, for specimens interpreted as equivocal after initial testing. Therefore, in one analysis, if the CT/GC assay result was negative, the identification assay results were not considered. In the alternate analysis, the results of the CT/GC assay were not considered for any samples; only the identification assay results, including equivocal zone resolution, were compared.
C. trachomatis culture. Culture for C. trachomatis was performed as described previously (15). McCoy cell monolayers in 96-well plates were inoculated with samples within 24 h of collection. Inoculated plates were fed a growth medium consisting of Eagle's minimal essential medium supplemented with 10% fetal calf serum, 4.4% glucose, and antibiotics. The plates were centrifuged for 1 h at 1,750 x g and were then incubated at 37°C for 48 to 72 h prior to staining. Cultures were stained with a fluorescein-labeled monoclonal antibody to C. trachomatis prepared in-house (15).
N. gonorrhoeae culture. Culture for N. gonorrhoeae was performed with modified Thayer-Martin agar in a 5% CO2 environment at 37°C. Cultures were assessed at 24 and 48 h for growth. The presence of N. gonorrhoeae isolates was confirmed if the colonies consisted of gram-negative diplococci, reacted to oxidase reagent, and stained positively with a fluorescent monoclonal antibody (Microtrak; Trinity Biotech, Bray, Ireland) specific for N. gonorrhoeae (17).
PCR. Specimens in CTM were stored at -70°C immediately following inoculation for culture. PCR was performed with a portion of the sample remaining after C. trachomatis culture. The instructions in the COBAS Amplicor CT/NG assay package insert were followed for all PCR tests. One hundred microliters of CTM was added to 100 µl of lysis buffer. After 10 min at room temperature, 200 µl of specimen diluent was added to the sample. Samples processed in this manner were then added to the master mixture reagent that contains enzymes and deoxynucleoside triphosphates in a buffer optimized for amplification. The samples were amplified and the DNA products were detected with the COBAS semiautomated system. Amplified products that bound to organism-specific oligonucleotides on magnetic beads were detected by using a biotin-streptavidin-mediated fluorescent readout. The internal control assay, which detects the amplification of a sequence of DNA included in the master mixture reagent, was used to rule out the possibility of inhibition of amplification. Interpretation of all PCR results was performed according to the algorithms in the manufacturer's package insert.
Statistical analyses.
The performance of HC2-RCS compared to that of HC2-M was analyzed by using the kappa statistic (
). This measure of agreement in categorical analyses is generally interpreted as follows: <0.20, poor agreement; 0.21 to 0.40, fair agreement; 0.41 to 0.60, moderate agreement; 0.61 to 0.80, good agreement; and >0.80 very good agreement (26). The infection status of patients from whom samples had been collected and placed in a repository could not be determined due to a lack of alternate samples for comparator testing. Therefore, the agreement between the two tests was the only measure of the performance of HC2-RCS calculated for these samples.
Since HC2-M has been cleared by the Food and Drug Administration for detection of these organisms from endocervical specimens and the objective of these studies was to evaluate the performance of HC2-RCS, HC2-M served as a component of the reference standard for the identification of infections in the prospective study. Patients were defined as being infected if they had a positive culture result or both a positive PCR result and a positive HC2-M result. Specimens positive only by HC2-M or only by PCR were considered false positive. McNemar's
2 test was used to compare the performance of HC2-RCS to those of both culture and PCR (
= 0.05).
| RESULTS |
|---|
|
|
|---|
scores ranged from 0.965 to 0.983 (Table 1). The agreement between the HC2-RCS and HC2-M formats was observed not only for interpreted results but also for the RLU/CO values observed for individual samples (Fig. 1).
|
|
Forty-five (13.6%) subjects were found to have C. trachomatis infection, which was defined by either a positive culture result or positive results by both PCR and the HC2-M CT-ID assay (Table 2). Had a positive HC2-M result been excluded from the definition of infection, two additional patients, positive by PCR alone, would have been defined as being infected with C. trachomatis. When the CT-ID assay alone was used, HC2-RCS yielded positive results for 43 of 45 (95.6%) samples. When the results for the specimens were interpreted on the basis of the initial result of the CT/GC assay and verification of positive results by the CT-ID assay, 42 of 45 (93.3%) samples were positive. The decrease in sensitivity was not significant (P > 0.5) when the CT/GC assay and identification verification algorithm were used instead of the CT-ID assay alone. There were no false-positive results by HC2-RCS. The use of the CT/GC assay did not change the interpretation of the results for any of the negative subjects. The sensitivity and specificity of the HC2-RCS CT-ID assay were equivalent to those of PCR (P > 0.5), and the sensitivity was significantly (P = 0.007) better than that of culture (Table 2).
|
|
Samples excluded from analysis. In the subset of 24 specimens that were not included in the analysis described above due to unavailable PCR results, 1 (4.2%) was positive for chlamydia by culture, while one other sample was positive for N. gonorrhoeae by culture. The prevalence of infection in this subset of samples excluded from the analysis is not statistically different from that in the remaining set of samples for either C. trachomatis or N. gonorrhoeae (P > 0.1 and P > 0.5, respectively). Each of the culture-positive samples was positive by the identification assays. All of the samples culture negative for N. gonorrhoeae were also negative by the GC-ID assay. Two samples culture negative for chlamydia were positive by the CT-ID assay, and the remaining 21 were negative by HC2-RCS. DNA amplification was inhibited for three of the samples (<1%) excluded from the analysis. Since the HC2 assay is a signal amplification assay, it does not include a control for DNA amplification inhibition. All of the samples that were inhibitory for PCR were culture negative for C. trachomatis and N. gonorrhoeae.
| DISCUSSION |
|---|
|
|
|---|
0.95 for all comparisons). This level of reproducibility is evidence of a highly robust test method for both C. trachomatis and N. gonorrhoeae. The proportion of samples that yielded results in the equivocal zone was low for both the high- and low-prevalence populations: 0.91 and 0.11%, respectively (data not shown). Retesting of these specimens in duplicate by the HC2 assay resulted in final interpretations for all samples and provided results in agreement with that of either culture or PCR for two of three samples. Resolution of equivocal results by retesting by either the HC2 assay or another test method may improve the specificity and reproducibility around the cutoff of the HC2 tests. Since only three samples had equivocal results by HC2-RCS in this study, further evaluation of the impact of retesting is necessary.
A limitation of the prospective study was the difficulty associated with defining infected patients. All evaluations of assays with potentially improved sensitivities compared to that of the "gold standard" assay face this issue. The bias inherent in discrepant analysis has been described mathematically (13; W. C. Miller, Editorial Response, Clin. Infect. Dis. 27:1186-1193, 1998). For this reason, discrepant analysis was not a part of the prospective study design. Instead, two reference tests, culture and PCR, were performed on all samples, and the results were included in the analysis of sensitivity and specificity. However, although we arbitrarily assigned a specificity of 100% to culture, this cannot be done for PCR. Therefore, we felt that the positive PCR results must be confirmed by another highly sensitive nucleic acid detection test in order to define infected patients with negative cultures. Darwin et al. (10) reported the relative sensitivities of HC2-M for the identification of C. trachomatis and N. gonorrhoeae to be 97.2 and 92.2%, respectively, compared to culture with discrepant resolution. The specificities were 99.2 and 99.8%, respectively, for identification of C. trachomatis and N. gonorrhoeae. Girdner et al. (12) also showed that HC2-M has 95.4% sensitivity and 99% specificity for the detection of C. trachomatis compared to the results of culture or direct fluorescent-antibody assay and PCR. On the basis of these performance parameters, we decided to use the results obtained from the identification assays of HC2-M to verify infection for culture-negative, PCR-positive patients. A positive HC2 assay result alone was not sufficient to define an infection. The specificity of HC2-RCS reported here was not affected by inclusion of a positive HC2-M result in the definition since samples positive by HC2-M alone were considered to be false positive in this analysis. Had we defined infection as a positive result by either culture or PCR alone, the performance estimates of the HC2 identification assays would not have changed significantly. The sensitivity of the CT-ID assay would have dropped from 93.3 to 91.5% (P > 0.2), and the sensitivity of the GC-ID assay would have dropped from 100 to 87.5% (P > 0.1); the specificity of either test would not have changed. However, exclusion of a positive HC2-M result from the definition of infection would have artificially set the specificity of PCR to 100%. The specificity measured by using the interpretation defined for this study resulted in PCR specificities of 99.3 and 99.0% for C. trachomatis and N. gonorrhoeae, respectively. These values are well within the range reported for this assay (12, 18, 22, 29, 30).
Another limitation of the data obtained by the HC2 assays is the storage and handling of the samples that was necessary for the multiple tests by both HC2-RCS and HC2-M. Samples from the repository were stored at -20°C for times longer than the limit recommended in the package insert. However, since this affected the sample itself, the effect should have been equivalent for both assay platforms. Since sensitivity and specificity calculations were not performed with data for these samples, there is no measurable impact as a result of extended storage. With the exception of samples with results in the equivocal zone, the samples used in the prospective study underwent three freeze-thaw cycles, the maximum allowable according to the package insert. Those with initial equivocal results underwent an additional thaw. However, this would be expected to reduce the sensitivities of the assays. Therefore, any impact from multiple freeze-thaw cycles would cause the data presented here to be a conservative estimate of the true performance of the HC2 assays.
For laboratories that test large volumes of samples for C. trachomatis and N. gonorrhoeae, the performances of the CT/GC assay and the subsequent organism-specific identification assays are of particular interest. The ability to perform a sensitive assay that will provide final results for the vast majority of specimens is highly desirable, especially for low-prevalence populations. In a population with an overall prevalence of infection with C. trachomatis and N. gonorrhoeae combined of 4%, as few as 15 of 352 samples would require testing by the CT-ID and GC-ID assays. In a population with a prevalence of nearly 18%, like that seen in the STD clinic population, as few as 62 of 352 samples would require additional testing. Since use of the CT/GC assay did not significantly affect the interpretation of results, each laboratory would have to evaluate the utility of screening by the CT/GC assay with confirmation by the organism-specific identification assays versus the use of the CT-ID and GC-ID assays as stand-alone tests. The HC2 assay system will allow laboratories the flexibility to choose the most economical testing scheme for the populations that they serve.
Due to the number of controls per plate, the HC2 assay may not be cost-effective in laboratories that perform less than 88 tests per run. However, this is similar to the number of controls required for other automated or semiautomated assays for C. trachomatis and N. gonorrhoeae combined. PCR by the COBAS Amplicor assay requires 8 controls for every 88 patient samples tested, the LCx assay (Abbott Diagnostics, Abbott Park, Ill.) requires 24 controls for every 72 samples, and the BD ProbeTec assay (Becton Dickinson Biosciences, Sparks, Md.) requires 6 controls for every 90 samples.
One RCS System instrument can test 352 samples plus 32 controls during a single work shift. During this testing procedure, 3.5 h of continuous hands-free time is available for technicians to perform other tasks. The availability of a high-throughput assay with excellent performance characteristics such as HC2-RCS is obviously desirable for large clinical laboratories and may influence programmatic decisions related to screening in less traditional settings. Although many studies have reported the cost-effectiveness of screening (5, 14, 19, 23), especially for chlamydial infections, these results are often based on targeted screening of high-risk populations. Use of the recommendations in both the United States and Europe and screening of women solely on the basis of age as a risk factor have not been found to be cost-effective when the lower-throughput systems available at present are used (20, 31; K. H. Webb, Letter to the Editor, Sex. Transm. Dis. 25:403-405, 1998). These analyses are strongly affected by the specificities of molecular assays. Increasing the specificity from 99.0 to 99.9% in a population with a 2% prevalence of infection reduced the cost per major outcome averted by up to 40% in one study (31). The impact on cost-benefit ratios of an assay that has a high sensitivity and a high specificity and that is designed to be compatible with high-throughput automation needs to be evaluated in future studies.
| FOOTNOTES |
|---|
| REFERENCES |
|---|
|
|
|---|
This article has been cited by other articles:
| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Antimicrob. Agents Chemother. | Clin. Microbiol. Rev. |
|---|---|
| Clin. Vaccine Immunol. | ALL ASM JOURNALS |
|---|