JCM Figure table search 04
Home Help [Feedback] [For Subscribers] [Archive] [Search] [Contents]
This Article
Right arrow Abstract Freely available
Right arrow Full Text (PDF)
Right arrow Alert me when this article is cited
Right arrow Alert me if a correction is posted
Services
Right arrow Similar articles in this journal
Right arrow Similar articles in PubMed
Right arrow Alert me to new issues of the journal
Right arrow Download to citation manager
Right arrowReprints and Permissions
Right arrow Copyright Information
Right arrow Books from ASM Press
Right arrow MicrobeWorld
Citing Articles
Right arrow Citing Articles via Google Scholar
Google Scholar
Right arrow Articles by Ridderhof, J. C.
Right arrow Articles by Robinson, P. H.
Right arrow Search for Related Content
PubMed
Right arrow PubMed Citation
Right arrow Articles by Ridderhof, J. C.
Right arrow Articles by Robinson, P. H.

 Previous Article  |  Next Article 

Journal of Clinical Microbiology, November 2003, p. 5258-5261, Vol. 41, No. 11
0095-1137/03/$08.00+0     DOI: 10.1128/JCM.41.11.5258-5261.2003
Copyright © 2003, American Society for Microbiology. All Rights Reserved.

Assessment of Laboratory Performance of Nucleic Acid Amplification Tests for Detection of Mycobacterium tuberculosis

John C. Ridderhof,1 Laurina O. Williams,1* Sue Legois,2 Peter A. Shult,2 Beverly Metchock,1 Louise N. Kubista,2 James H. Handsfield,1 Ronald J. Fehd,1 and Pamela H. Robinson1

Centers for Disease Control and Prevention, Atlanta, Georgia 30333,1 Wisconsin State Laboratory of Hygiene, Madison, Wisconsin 537062

Received 12 July 2002/ Returned for modification 6 October 2002/ Accepted 22 August 2003


   ABSTRACT
Top
 Abstract
Text
 References
 
During implementation of the Centers for Disease Control and Prevention's Mycobacterium tuberculosis nucleic acid amplification (NAA) evaluation program, 27.1% of participants used the same biological safety cabinet for NAA and specimen processing; 28.8% reported not using unidirectional workflow. An association between false positives and adverse responses to quality assurance questions (P = 0.04) illustrated the need for following NCCLS recommendations.


   TEXT
Top
 Abstract
 Text
References
 
Procedures for rapid laboratory testing for diagnosing tuberculosis (TB) are a major component of public health policy for controlling TB in the United States (17). The nucleic acid amplification (NAA) test for direct detection of Mycobacterium tuberculosis in patient-derived specimens provides a rapid result to determine whether a patient has TB. Information from the M. tuberculosis NAA result may contribute to patient diagnosis and to decisions involving respiratory isolation within a hospital. Two M. tuberculosis NAA tests, Gen-Probe MTD and the Roche Amplicor MTB, were approved by the U.S. Food and Drug Administration in 1995 for use with acid-fast bacilli smear-positive respiratory specimens (3). In 2000, the modified Gen-Probe MTD test was approved for use with smear-negative respiratory specimens (4). The U.S. Department of Health and Human Services has advocated increased use of M. tuberculosis NAA tests for rapid diagnosis and control of TB in the Healthy People 2010 Objectives (18).

Commercial development of the Gen-Probe MTD and the Roche Amplicor MTB provides access to M. tuberculosis NAA tests for routine diagnostic testing in laboratories which may not have had experience with molecular testing. The NAA tests, however, require specific quality assurance (QA) practices to prevent contamination. While the performance characteristics of M. tuberculosis NAA tests were documented previously (1, 2, 6-11, 15, 16), there is little knowledge of laboratory practices and performance associated with these tests in routine diagnostic settings (13, 14).

To assess laboratory practices and performance with M. tuberculosis NAA tests, the Centers for Disease Control and Prevention (CDC) implemented a M. tuberculosis NAA testing performance evaluation program in 1997. This voluntary program uses samples containing M. tuberculosis and nontuberculous mycobacteria (NTM), providing an anonymous challenge for laboratories (http://www.phppo.cdc.gov/mpep/mtbnaa.asp). An aggregate report of results is provided to participants with analyses of common problems. We describe here the laboratory participants, laboratory practices for QA reported, aggregate testing performance, and analyses of the association between performance and QA practices for the combined results from two 1997 shipments.

Test samples were developed from patient-derived isolates of M. tuberculosis and NTM (Mycobacterium kansasii and Mycobacterium gordonae). Stock suspensions were prepared from 8-day growth on Lowenstein-Jensen medium and adjusted to an absorbency of 0.2 at 420 nm. Dilutions were made in either sterile water or phosphate buffer (0.15 M, pH 6.8). Organism counts were measured using a Bryte H5 flow cytometer (Bio-Rad Laboratories). Viability was confirmed by plate counts on Middlebrook 7H10 agar. Sensitivity for M. tuberculosis, using the 50-µl analytic test volume recommended by Gen-Probe MTD, was determined using samples with concentrations ranging from 1.5 x 106 organisms to less than 1 organism per ml. Sample stability at various concentrations was tested by holding unfrozen samples before assay at room temperature for 2 to 3 days, at -70°C for 12 weeks, and for 4, 6, or 8 days at both 4°C and at room temperature after thawing. In each shipment, a total of five samples including aliquots of M. tuberculosis and M. gordonae or M. kansasii were sent to participating laboratories. Participants were instructed to analyze the samples within 72 h.

Anonymous results were analyzed by the CDC and the Wisconsin State Laboratory of Hygiene. Univariate analysis of data from two shipments (April 1997 and September 1997) was conducted for laboratory demographics, methods, and appropriate use of the biological safety cabinet (BSC). The cumulative qualitative test results for the 10 samples were collated.

For assessment of the relationship between QA practices and specificity for M. tuberculosis NAA testing, results for the M. gordonae samples were compared with a composite score of the laboratory's responses to questions about QA practices. Only the results from laboratories using the Gen-Probe MTD were included due to low numbers for other methods. Participants were asked to report on uses of their BSC for M. tuberculosis NAA testing. They were also asked whether they used unidirectional workflow. Responses to each question were scored in order from the least-preferred to most-preferred QA practices. Composite scores were then compared with sample test results using Fischer's exact test. Adverse responses to QA questions were as follows: use of the same BSC for M. tuberculosis NAA testing and for M. tuberculosis specimen processing for other uses; not using unidirectional workflow (or not knowing if used). Bivariate analysis at the 95% confidence level was used to determine whether adverse answers to QA questions were associated with false-positive results.

Of the 86 participants in the M. tuberculosis NAA performance evaluation program in 1997, 55% (47 of 86) were hospital laboratories, 27% (23 of 86) were health department laboratories, 15% (13 of 86) were independent laboratories, and 2% (2 of 86) were other types of laboratories. Most laboratories (74% [64 of 86]) performed the Gen-Probe MTD test. Fourteen percent (12 of 86) of the laboratories performed the Roche Amplicor MTB test, 12% (10 of 86) used in-house methods, and 2% (2 of 86) used other methods.

A wide distribution in weekly patient specimen volume was observed among participants (Fig. 1), with many laboratories testing <1 specimen per week. Overall sensitivity for all methods for detecting M. tuberculosis was 97.9% (417 of 426). For M. gordonae samples, overall specificity was 93.5% (314 of 336); however, overall specificity for the M. kansasii sample was 71.6% (58 of 81).



View larger version (13K):
[in this window]
[in a new window]
  		FIG. 1. Number of patient-derived specimens tested using M. tuberculosis NAA tests between 1 January 1997 and 31 December 1997.

 
Approximately 27% of participants reported that the BSC used for M. tuberculosis NAA testing was also used for TB specimen processing (Fig. 2), and 26% reported using the BSC for other purposes. Additionally, 34% and 24% of participants in the April 1997 and September 1997 shipments, respectively, reported either not using unidirectional workflow or not knowing whether unidirectional workflow was used in performing M. tuberculosis NAA testing (Fig. 3). Among laboratories using Gen-Probe MTD that had false-positive M. tuberculosis NAA results with M. gordonae samples, there was at least one response to a QA question that indicated a practice that is not recommended (Table 1). In comparing false-positive qualitative results with a composite score of adverse responses to QA questions regarding BSC use and unidirectional workflow, false-positive results were significantly associated with an adverse quality assurance score (P = 0.04). When inconclusive results were included as incorrect responses, the association was weakened (P = 0.07). A total of 10 false-positive results were reported by nine different laboratories for both shipments.



View larger version (32K):
[in this window]
[in a new window]
  		FIG. 2. Uses of the BSC for M. tuberculosis NAA and for other purposes by laboratories participating in the M. tuberculosis NAA PE in April and September 1997. *, numbers at the end of each bar represent the frequency of results; **, mycobacterial culture identification or mycology or virology testing.

 


View larger version (28K):
[in this window]
[in a new window]
  		FIG. 3. Use of unidirectional workflow by laboratories participating in the M. tuberculosis NAA PE in April and September 1997. *, numbers at the end of each bar represent the frequency of results.

 

View this table:
[in this window]
[in a new window]
  		TABLE 1. QA survey of laboratories reporting false-positive results (Gen-Probe MTD)

 
The CDC M. tuberculosis NAA performance evaluation program participants represent a large cohort of laboratories using commercial M. tuberculosis NAA tests in the United States. Cross-reactivity with M. kansasii has been previously documented with Gen-Probe MTD (10). The Gen-Probe MTD has since been modified by the manufacturer, which may have decreased cross-reactivity with NTM. Results from a subsequent shipment containing a strain of Mycobacterium celatum, which has also documented cross-reactivity, indicated improved test performance with the modified Gen-Probe MTD (19). However, false-positive and inconclusive results with M. gordonae samples may indicate cross-contamination. Results from the first two shipments indicate concerns about specific M. tuberculosis NAA testing QA practices. Although the NCCLS recommends separation of equipment and work areas for NAA testing (12), some participants reported processing specimens for M. tuberculosis smear or culture in the same BSC used for M. tuberculosis NAA testing and either not using unidirectional workflow or not knowing whether unidirectional workflow is used. False-positive results with Gen-Probe MTD were significantly associated (P = 0.04) with a decreased QA score based on participant responses about use of the BSC and unidirectional workflow. There are indications from previous studies that false-positive results due to cross-contamination may be a problem with some methods (2, 5, 6). These analyses highlight potential problems with the use of commercial NAA tests in laboratories that may not have the equipment, facilities, or special QA expertise which NAA testing requires. Use of molecular diagnostics requires special attention to the configuration of workflow and multiple uses of laboratory space and equipment, which may require modifications in laboratory design and additional resources. Of special concern is the perception that the commercialization and Food and Drug Administration clearance of test kits provide laboratories protection from the same cross-contamination problems that are inherent with in-house-developed test methods.

These results represent the first two shipments of specimens. In subsequent shipments of this program, fewer false-positive results were observed, although there are remaining concerns about some of the QA practices (data not shown). This improved performance may possibly be due to increased attention to contamination based on the reports from this program and increased experience with test methods. The potential for cross-contamination with commercial NAA tests, however, still exists. These results demonstrate that laboratories should be vigilant and follow standard QA practices. The false-positive test results encountered in this program also support the need to interpret positive M. tuberculosis NAA test results with caution, especially if the specimen is acid-fast bacilli smear negative (4). Based on results, we recommend the following: that laboratories follow NCCLS recommendations for NAA testing; that manufacturers include QA recommendations (e.g., unidirectional workflow and separate equipment and areas) in package inserts; and that proficiency testing organizations increase the number of samples with NTM in their M. tuberculosis NAA testing programs to assist laboratories in detecting problems with contamination.


   ACKNOWLEDGMENTS
 
Use of trade names and commercial sources is for identification only and does not imply endorsement by the Public Health Service or by the U.S. Department of Health and Human Services.


   FOOTNOTES
 
* Corresponding author. Mailing address: Division of Laboratory Systems, Centers for Disease Control and Prevention, Mailstop G25, 4770 Buford Highway, NE, Atlanta, GA 30341. Phone: (770) 488-8130. Fax: (770) 488-8275. E-mail: low1{at}cdc.gov. Back


   REFERENCES
Top
 Abstract
 Text
 References
 

  1. Bergmann, J., G. Yuoh, G. Fish, and G. Woods. 1999. Clinical evaluation of the enhanced Gen-Probe amplified Mycobacterium tuberculosis direct test for rapid diagnosis of tuberculosis in prison inmates. J. Clin. Microbiol. 37:1419-1425.[Abstract/Free Full Text]
  2. Cartuyvels, R., C. De Ridder, S. Jonckheere, L. Verbist, and J. Van Eldere. 1996. Prospective clinical evaluation of Amplicor Mycobacterium tuberculosis PCR test as a screening method in a low-prevalence population. J. Clin. Microbiol. 34:2001-2003.[Abstract]
  3. Centers for Disease Control and Prevention. 1996. Nucleic acid amplification tests for tuberculosis. Morb. Mortal. Wkly. Rep. 45:950-952.[Medline]
  4. Centers for Disease Control and Prevention. 2000. Update: nucleic acid amplification tests for tuberculosis. Morb. Mortal. Wkly. Rep. 49:593-594.[Medline]
  5. Clarridge, J. E. III, R. M. Shawar, T. M. Shinnick, and B. B. Plikaytis. 1993. Large-scale use of polymerase-chain reaction for detection of Mycobacterium tuberculosis in a routine mycobacteriology laboratory. J. Clin. Microbiol. 31:2049-2056.[Abstract/Free Full Text]
  6. Della-Latta, P., and S. Whittier. 1998. Comprehensive evaluation of performance, laboratory application, and clinical usefulness of two direct amplification technologies for the detection of Mycobacterium tuberculosis complex. Am. J. Clin. Pathol. 110:301-310.[Medline]
  7. Gamboa, F., G. Fernandez, E. Padilla, J. Manterola, J. Lonca, P. J. Cardona, L. Matas, and V. Ausina. 1998. Comparative evaluation of initial and new versions of the Gen-Probe Amplified Mycobacterium tuberculosis Direct Test for direct detection of Mycobacterium tuberculosis in respiratory and non-respiratory specimens. J. Clin. Microbiol. 36:684-689.[Abstract/Free Full Text]
  8. Ichiyama, S., Y. Iinuma, Y. Tawada, S. Yamori, Y. Hasegawa, K. Shimokata, and N. Nakashima. 1996. Evaluation of Gen-Probe Amplified Mycobacterium tuberculosis Direct Test and Roche PCR-microwell plate hybridization method (Amplicor Mycobacterium) for direct detection of mycobacteria. J. Clin. Microbiol. 34:130-133.[Abstract]
  9. Jonas, V., M. J. Alden, J. I. Curry, K. Kamisango, C. A. Knott, R. Lankford, J. M. Wolfe, and D. F. Moore. 1993. Detection and identification of Mycobacterium tuberculosis directly from sputum sediments by amplification of rRNA. J. Clin. Microbiol. 31:2410-2416.[Abstract/Free Full Text]
  10. Jorgensen, J. H., J. R. Salinas, R. Paxson, K. Magnon, J. E. Patterson, and T. F. Patterson. 1999. False-positive Gen-Probe direct M. tuberculosis amplification test results for patients with pulmonary M. kansasii and M. avium infections. J. Clin. Microbiol. 37:175-178.[Abstract/Free Full Text]
  11. Miller, N., S. Hernandez, and T. Cleary. 1994. Evaluation of Gen-Probe Amplified Mycobacterium tuberculosis Direct Test and PCR for direct detection of Mycobacterium tuberculosis in clinical specimens. J. Clin. Microbiol. 32:393-397.[Abstract/Free Full Text]
  12. National Committee for Clinical Laboratory Standards. 1995. Molecular diagnostic methods for infectious diseases: approved guidelines (NCCLS document MM3-A, 1995). National Committee for Clinical Laboratory Standards, Wayne, Pa.
  13. Noordhoek, G. T., J. D. A. van Embden, and A. H. J. Kolk. 1996. Reliability of nucleic acid amplification for detection of Mycobacterium tuberculosis: an international collaborative quality control study among 30 laboratories. J. Clin. Microbiol. 34:2522-2525.[Abstract]
  14. Noordhoek, G. T., A. H. J. Kolk, G. Bjune, D. Catty, J. W. Dale, P. E. M. Fine, P. Godfrey-Faussett, S.-N. Cho, T. Shinnick, S. B. Svenson, S. Wilson, and J. D. A. van Embden. 1994. Sensitivity and specificity of PCR for detection of Mycobacterium tuberculosis: a blind comparison study among seven laboratories. J. Clin. Microbiol. 32:277-284.[Abstract/Free Full Text]
  15. Pfyffer, G. E., P. Kissling, E. M. I. Jahn, H.-M. Welscher, M. Salfinger, and R. Weber. 1996. Diagnostic performance of amplified Mycobacterium tuberculosis direct test with cerebrospinal fluid, other nonrespiratory, and respiratory specimens. J. Clin. Microbiol. 34:834-841.[Abstract]
  16. Piersimoni, C., A. Callegaro, D. Nista, S. Bornigia, F. de Conti, G. Santini, and G. de Sio. 1997. Comparative evaluation of two commercial amplification assays for direct detection of Mycobacterium tuberculosis complex in respiratory specimens. J. Clin. Microbiol. 35:193-196.[Abstract]
  17. Tenover, F. C., J. C. Crawford, R. E. Huebner, L. J. Geiter, C. R. Horsburgh, Jr., and R. C. Good. 1993. The resurgence of tuberculosis: is your laboratory ready? J. Clin. Microbiol. 31:767-770.[Free Full Text]
  18. U. S. Department of Health and Human Services. 2000. Healthy people 2010, 2nd ed. With understanding and improving health objectives for improving health, vol. 1, section 14-14. U.S. Government Printing Office, Washington, D.C.
  19. Williams, L. O., and J. C. Ridderhof.1998. Analysis of performance evaluation results for M. tuberculosis nucleic acid amplification testing reported to the Centers for Disease Control and Prevention by participating laboratories, biannual report. Division of Laboratory Systems, Public Health Practice Program Office, Centers for Disease Control and Prevention, Atlanta, Ga.

Journal of Clinical Microbiology, November 2003, p. 5258-5261, Vol. 41, No. 11
0095-1137/03/$08.00+0     DOI: 10.1128/JCM.41.11.5258-5261.2003
Copyright © 2003, American Society for Microbiology. All Rights Reserved.




This Article
Right arrow Abstract Freely available
Right arrow Full Text (PDF)
Right arrow Alert me when this article is cited
Right arrow Alert me if a correction is posted
Services
Right arrow Similar articles in this journal
Right arrow Similar articles in PubMed
Right arrow Alert me to new issues of the journal
Right arrow Download to citation manager
Right arrowReprints and Permissions
Right arrow Copyright Information
Right arrow Books from ASM Press
Right arrow MicrobeWorld
Citing Articles
Right arrow Citing Articles via Google Scholar
Google Scholar
Right arrow Articles by Ridderhof, J. C.
Right arrow Articles by Robinson, P. H.
Right arrow Search for Related Content
PubMed
Right arrow PubMed Citation
Right arrow Articles by Ridderhof, J. C.
Right arrow Articles by Robinson, P. H.

Home Help [Feedback] [For Subscribers] [Archive] [Search] [Contents]
Antimicrob. Agents Chemother. Clin. Microbiol. Rev.
Clin. Vaccine Immunol. ALL ASM JOURNALS