Skip to main page content

• HOME
• Current Issue
• Archive
• Alerts
• About ASM
• Contact us
• Tech Support
• Journals.ASM.org

This Article Abstract Full Text (PDF) Alert me when this article is cited Alert me if a correction is posted Services Similar articles in this journal Similar articles in PubMed Alert me to new issues of the journal Download to citation manager Reprints and Permissions Copyright Information Books from ASM Press MicrobeWorld Citing Articles Citing Articles via Google Scholar Google Scholar Articles by van Belkum, A. Articles by van Leeuwen, W. B. Search for Related Content PubMed PubMed Citation Articles by van Belkum, A. Articles by van Leeuwen, W. B.

 Previous Article  |  Next Article 

Journal of Clinical Microbiology, August 2007, p. 2698-2700, Vol. 45, No. 8
0095-1137/07/$08.00+0     doi:10.1128/JCM.00759-07
Copyright © 2007, American Society for Microbiology. All Rights Reserved.

Quality Control of Direct Molecular Diagnostics for Methicillin-Resistant Staphylococcus aureus

Alex van Belkum,^1* Hubert G. M. Niesters,^2, William G. MacKay,³ and Willem B. van Leeuwen¹

Department of Medical Microbiology and Infectious Diseases, Unit Research and Development,¹ Department of Virology, Erasmus MC, 's-Gravendijkwal 230, 3015 CE Rotterdam, The Netherlands,² the Neutral Office, Quality Control for Molecular Diagnostics, Block 4, Kelvin Campus, West of Scotland Science Park, Glasgow G20 OSP, Scotland³

Received 10 April 2007/ Returned for modification 12 May 2007/ Accepted 11 June 2007

Top ABSTRACT TEXT REFERENCES

 
Ten samples containing various amounts of methicillin-resistant Staphylococcus aureus (MRSA), methicillin-susceptible S. aureus, methicillin-resistant Staphylococcus epidermidis (MRSE), and combinations thereof were distributed to 51 laboratories for molecular diagnostics testing. Samples containing 10² to 10³ MRSA cells were frequently reported to be negative. MRSE samples were scored as negative by all commercial tests but by only two out of three in-house tests.

Top ABSTRACT TEXT REFERENCES

 
Methicillin-resistant Staphylococcus aureus (MRSA) requires timely detection in order to prevent infection and nosocomial transmission (1, 4). Classical microbiologists routinely use culture-based enrichment assays as the basis for detection and subsequent identification. The inclusion of an elevated salt concentration and specific antibiotics in the growth medium allows for highly specific detection (8). However, culture requires prolonged incubation periods, and in general, confirmatory assays are required upon positive culture. Molecular assays may offer benefits over more traditional culture-based assays, such as reduced time to identification and better specificity and sensitivity. The commercial diagnostics industry has introduced a range of molecular assays in recent years. However, there is little information available on their performance, and some studies suggest that they may suffer from reduced sensitivity as a result of sample inhibition (3). Specificity may also be an issue as the presence of methicillin-resistant coagulase-negative staphylococci, including methicillin-resistant Staphylococcus epidermidis (MRSE), may result in false-positive results. Given the number of commercially available test systems, the clinical impact of MRSA, and the need for timely diagnostics, a multicenter external quality assessment (EQA) study on the efficiency and efficacy of molecular testing for MRSA was initiated by Quality Control for Molecular Diagnostics (QCMD) (www.qcmd.org).

The QCMD MRSA EQA panel samples were distributed in October 2006 to 51 participating laboratories from 11 countries, along with detailed instructions on how to process the panel samples. Participants were given 6 weeks to report their results back to the QCMD Neutral Office by using an online data collection system.

The QCMD 2006 MRSA panel consisted of five samples containing various amounts of MRSA, three samples containing various amounts of staphylococci other than S. aureus, and one sample containing Escherichia coli (Table 1). The contents of the samples were quantified on the basis of culture and molecular testing results. The IDI-MRSA test (Becton Dickinson) was employed in combination with the Sigma plant DNA isolation kit according to the manufacturers' instructions.

View this table: [in this window] [in a new window]

TABLE 1. Panel composition and results of independent testinga

Out of the 51 participants, 46 (90%) responded. Nonrespondents indicated technical problems (n = 2) or "test under development" (n = 2) as the reason for not returning results. Overall, 58 data sets were returned, 55 of which included qualitative data only; three labs reported both qualitative and quantitative data. All participants received the expected results following the close of the program. Subsequently, the QCMD Neutral Office analyzed the data, which was released to participants in the form of a detailed EQA final report.

Most of the real-time data were generated with the Roche LightCycler system (n = 14), Roche LightCycler 2.0 (n = 6), and the Corbett Research Rotor-Gene 3000 (n = 5) and Applied Biosystems systems (the ABI 7500 real-time PCR system [n = 6], the ABI PRISM 7000 sequence detection system [n = 5], and the ABI PRISM 7900 sequence detection system [n = 1]). Two Bio-Rad machines were included, and the Cepheid Smart Cycler II system, Roche LightCycler 480, and the Stratagene MxP3000 real-time system each provided a single data set. For the in-house conventional PCRs, the Eppendorf MasterCycler, the MWG AG Biotech Primus 96, and the Perkin-Elmer 9600 were used. The diversity of the equipment covers the spectrum commercially available systems quite well.

QCMD used a simple scoring system for qualitative EQA data, which was as follows: 2 points for a correct result and 0 points for all other results (including "not determined" and "equivocal"). The results obtained are summarized in Table 2. Results for the panel sample with the highest number of MRSA cells (MRSA06-08) were reported correctly in 97% of the data sets. This finding indicates that the usage of molecular tests to follow up positive cultures is reliable: all the tests can be used for culture confirmation. However, the samples containing smaller amounts of MRSA (MRSA06-09 and MRSA06-06) had levels that were below the limit of detection of most participants' assays. Only 12 and 52% of the data sets included correct results for these samples. MRSE sample MRSA06-07 was correctly reported as negative for MRSA by all commercial PCR tests but incorrectly reported as positive in 33% (1 of 3) of data sets generated with conventional in-house assays and 21% (9 of 43) of data sets generated using real-time in-house PCR. The methicillin-susceptible S. aureus (MSSA)-containing sample MRSA06-10 was scored incorrectly by both commercial and in-house tests (17% of conventional PCR tests and 9% of real-time PCR tests), indicating moderate specificity. These levels of false positives underscore the need for improved specificity of these MRSA tests. Even the E. coli specimen could not be adequately tested (results were indeterminate in 7% [4 of 58] of data sets). Table 3 summarizes the performance scores for the various PCR platforms. The performance of the commercial conventional PCRs and the in-house real-time PCRs can be considered satisfactory. However, only three participants attained the maximum score of 20.

View this table: [in this window] [in a new window]

TABLE 2. Technology types and numbers of correct qualitative results per panel sample

View this table: [in this window] [in a new window]

TABLE 3. Qualitative performance scores per technology typea

Quality control of molecular diagnostics tools is important in order to maintain high-quality clinical care in medical institutions. Multicenter studies on the sensitivity and specificity of several bacterial and viral DNA tests have been performed in the past, and the outcomes of such studies have provided important information on performance (2, 5, 6, 7). Here we report on an EQA study of one of the most important nosocomial bacterial pathogens, MRSA. In conclusion, the molecular identification of MRSA by using samples with high CFU counts is reliable and can be implemented in the laboratory setting with confidence. Essentially, all testing formats perform equally well. However, for direct molecular diagnostics, we have to conclude that the present array of tests do not meet the clinical quality criteria. The sensitivity of many tests is (too) low, and the specificity needs to be improved. The reasons for poor assay sensitivity may be grounded in the common practice of preenriching samples by culture before confirmation by molecular assays. Preenrichment may lead to concentrations of MRSA in excess of 10¹⁰ CFU/ml, which is greater than the concentrations of MRSA likely to be encountered in an unmodified patient sample (and those in this EQA panel). Some assays may have been designed specifically for this higher target concentration range, leading to poor levels of sensitivity. The preenrichment approach also reduces one of the key improvements offered by molecular assays, which is more rapid diagnosis. The sensitivity of molecular assays for the detection of MRSA will improve only if the molecular diagnostics community moves away from preenrichment and tests directly from the clinical sample. Performance would also be improved by the provision of proper quality control materials and international standards for MRSA testing.

Although the stage has been set for direct clinical detection of MRSA, our data here indicate that the present testing systems are insufficient. And clinical practice may even be worse: here we used clean samples, and in the case of samples such as urine, blood, and sputa, inhibitory compounds within these samples may deteriorate the test performance. These data clearly support the need for improvements in the molecular detection of MRSA.

 
The QCMD program is organized in collaboration with the European Society for Clinical Virology and the European Society for Clinical Microbiology and Infectious Diseases. Panels were produced in the Department of Medical Microbiology and Infectious Diseases of Erasmus MC, Rotterdam, The Netherlands. Nothing in this report may be reproduced without permission of the QCMD Executive Office.

 
* Corresponding author. Mailing address: Department of Medical Microbiology and Infectious Diseases, Unit Research and Development, Erasmus MC, 's-Gravendijkwal 230, 3015 CE Rotterdam, The Netherlands. Phone: 00-31-10-4635813. Fax: 00-31-10-4633875. E-mail: a.vanbelkum{at}erasmusmc.nl

Published ahead of print on 20 June 2007.

Present address: Department of Medical Microbiology, Section Clinical Virology Hanzeplein 1, University Medical Center Groningen, P.O. Box 30 001, 9700 RB Groningen, The Netherlands.

Top ABSTRACT TEXT REFERENCES

 

1
1. Brown, D. F., D. I. Edwards, P. M. Hawkey, D. Morrison, G. L. Ridgway, K. J. Towner, M. W. Wren, and the Joint Working Party of the British Society for Antimicrobial Chemotherapy, Hospital Infection Society, Infection Control Nurses Association. 2005. Guidelines for the laboratory diagnosis and susceptibility testing of methicillin resistant Staphylococcus aureus (MRSA). J. Antimicrob. Chemother. 56:1000-1018.[Abstract/Free Full Text]
2
2. Donoso Mantke, O., S. W. Aberle, T. Avsic-Zupanc, M. Labuda, and M. Niedrig. 2007. Quality control assessment for the PCR diagnosis of tick-borne encephalitis virus infections. J. Clin. Virol. 38:73-77.[CrossRef][Medline]
3
3. Hodgson, J., M. Zuckerman, and M. Smith. 2007. Development of a novel internal control for a real time PCR for HSV DNA types 1 and 2. J. Clin. Virol. 38:217-220.[CrossRef][Medline]
4
4. Mellmann, A., A. W. Friedrich, N. Rosenkotter, J. Rothganger, H. Karch, R. Reintjes, and D. Harmsen. 2006. Automated DNA sequence based early warning system for the detection of methicillin-resistant Staphylococcus aureus outbreaks. PLoS Med. 3:e33.[CrossRef][Medline]
5
5. Noordhoek, G. T., S. Mulder, P. Wallace, and A. M. Van Loon. 2004. Multicentre quality control study for detection of Mycobacterium tuberculosis in clinical samples by nucleic acid amplification methods. Clin. Microbiol. Infect. 10:295-301.[CrossRef][Medline]
6
6. Templeton, K. E., C. B. Forde, A. M. Loon, E. C. Claas, H. G. M. Niesters, P. Wallace, and W. F. Carman. 2006. A multi-centre pilot proficiency programme to assess the quality of molecular detection of respiratory viruses. J. Clin. Virol. 35:51-58.[CrossRef][Medline]
7
7. Valentine-Thon, E. 2002. Quality control in nucleic acid testing: where do we stand? J. Clin. Virol. 25(Suppl.):S13-S21.
8
8. Wertheim, H., H. A. Verbrugh, C. Van Pelt, P. De Man, A. Van Belkum, and M. C. Vos. 2001. Improved detection of methicillin-resistant Staphylococcus aureus using phenyl mannitol broth containing aztreonam and ceftizoxime. J. Clin. Microbiol. 39:2660-2662.[Abstract/Free Full Text]

---

Journal of Clinical Microbiology, August 2007, p. 2698-2700, Vol. 45, No. 8
0095-1137/07/$08.00+0     doi:10.1128/JCM.00759-07
Copyright © 2007, American Society for Microbiology. All Rights Reserved.

This Article Abstract Full Text (PDF) Alert me when this article is cited Alert me if a correction is posted Services Similar articles in this journal Similar articles in PubMed Alert me to new issues of the journal Download to citation manager Reprints and Permissions Copyright Information Books from ASM Press MicrobeWorld Citing Articles Citing Articles via Google Scholar Google Scholar Articles by van Belkum, A. Articles by van Leeuwen, W. B. Search for Related Content PubMed PubMed Citation Articles by van Belkum, A. Articles by van Leeuwen, W. B.