We read with interest the article by Murk et al. (4). The authors conclude that a simple overnight preenrichment step improves the detection of extended-spectrum-beta-lactamase (ESBL)-positive strains and permits earlier recognition and isolation of patients who carry these strains. We would like to address some concerns we have about the validity of these data.
First, it is of major importance to use controlled and standardized conditions in the evaluation of culture-based screening protocols. During a 2-month period, 500 surveillance specimens were collected from 88 mechanically ventilated intensive care unit (ICU) patients. The patients' swabs were first streaked on a beta-lactamase screening agar (BLSE) (AES Chemunex, Bruz cedex, France) and then inserted into 5 ml of antibiotic-free Trypticase soy broth (TSB) for overnight incubation at 37°C. The following day, 100 μl of the enriched samples was subcultured onto BLSE. Although the inoculum size is a very important parameter in this type of evaluation, the authors did not use the same inoculum for both methods. Therefore, a potentially larger inoculum in the TSB could have influenced the sensitivities of the methods evaluated, e.g., an increased sensitivity for the enrichment protocol.
Second, the authors did not use a defined, genotypic gold standard for the detection of ESBL. For the confirmation of ESBL-producing isolates, the double-disc synergy test and Etest were used. One patient harbored an ESBL-positive strain of Achromobacter xylosoxidans, but to our knowledge, there are no guidelines for phenotypic ESBL detection for this species. Resistance mechanisms other than ESBL, such as hyperproduction of SHV-1, may give false-positive confirmation tests. Therefore, confirmation of phenotypic results by genotypic characterization of resistance mechanisms should have been confirmed by PCR assays targeting bla genes with amplicon sequencing.
Third, the authors showed in their analyses of the 500 surveillance specimens collected from 88 ICU patients that with enrichment, the number of cultures that yielded ESBL-positive bacteria (n = 20) was twice that of cultures without enrichment (n = 10), which is a statistically significant difference (McNemar's test; P = 0.006).
However, when paired binary responses (cultured with enrichment versus cultured without enrichment) are compared using McNemar's test, it is assumed that the paired responses are independent of each other (3). Subsequently, the test accounts for the correlation within the paired responses in the variance of the test statistic.
The 500 paired responses of the surveillance specimens do not seem to fulfill this independence assumption, as individual patients have contributed more than one paired response (patient 1 contributed 4 cultured swabs; patient 2, 14 swabs; patient 3, 19 swabs; patient 4, 4 swabs; patient 5, 2 swabs; patient 6, 23 swabs; patient 7, 6 swabs; patient 8, 9 swabs; patient 9, 8 swabs). When a patient contributes more than one paired response to the study sample, the variability of the observations for the patient is less than the variability among different patients. Analyzing these 500 paired responses as independent, without accounting for the clustering of paired responses (where the cluster is the patient), will result in an underestimated standard error and P value (1, 2). It would be interesting to find out if the highly significant difference in ESBL-positive cultures with and without the enrichment step would remain after accounting for the apparent clustering that is present in this sample of paired responses. There are several statistical methods that could be used in the analysis of clustered matched-pair data (3).
Determining the optimal ESBL screening protocol is clearly of the utmost importance. Studies like those conducted by Murk et al. will hopefully set the basis for carrying out prospective research specially designed to avoid recognizable biases.
- Copyright © 2009 American Society for Microbiology
REFERENCES
Authors' Reply
We thank Dr. Diederen and Dr. Euser for their interest in our paper and are happy to engage in a discussion about their criticisms.
A first comment concerns the inoculum that we used for the cultures with and without enrichment. Diederen and Euser, however, have misread this part of the Materials and Methods section, because what we actually did is stated on page 1886: “On the day that the surveillance cultures were obtained, the patient's swabs were first streaked on BLSE agar and then inserted into 5 ml of antibiotic-free TSB for overnight incubation at 37°C. The next day, the swabs in the TSB-enriched cultures were streaked on BLSE plates” (3). Thus, we used the same swab for both culture methods.
A second point of criticism is that we did not use a defined, genotypic gold standard for the detection of ESBLs but confirmed ESBL production with the double-disc synergy test and Etest. Indeed, we used phenotypic tests to confirm ESBL production; in this we were following the leading guidelines (1). We used the same methods to confirm ESBL production for all isolates, whether they were detected with or without prior enrichment. With respect to our positive cultures with Enterobacteriaceae, the typical ESBL phenotype of these bacteria left little room to doubt the phenotypic confirmation, although it would have been interesting to identify and sequence the responsible genes. We agree with the commentators that genotypic characterization would have been useful in the culture with Achromobacter xylosoxidans, because so little is known about the presence and detection of ESBLs in these microorganisms. The outcome of the genotypic characterization, however, is not essential for the overall conclusions of our study, because the difference between both culture methods still retains statistical significance if the cultures of the patient that yielded A. xylosoxidans are left out (P = 0.012, McNemar's test; see also the discussion about the statistics below).
Finally, Diederen and Euser argue that McNemar's test is not appropriate to analyze our data. Our decision to use this test was motivated by the assumption that differences in performance between the two culture methods were due to sample-specific features rather than patient characteristics. Various extensions of McNemar's test have been proposed for situations where this assumption of independence between samples does not necessarily hold. Most of these, however, have reduced power when success probability, i.e., the probability of ESBL detection, varies between clusters (patients). Also, these adjustments make additional assumptions about the correlation pattern of the data that are less transparent and harder to assess than the one assumption underlying McNemar's test. For these reasons, we restrained from using an adjusted McNemar test. As an alternative, we might have used a test developed by Obuchowski (2), which is derived from sampling theory and avoids the above-mentioned limitations. The observation that the number of samples per patient was highly variable also suggests that Obuchowski's test is in fact more appropriate to our data than the adjustments to McNemar's test proposed by Diederen and Euser, as it retains power with increasing cluster size variability.
If we correct for the fact that multiple samples were derived from individual patients according to Obuchowski's test, the P value is only marginally inflated and retains statistical significance (P = 0.023) even if we leave out of the analysis the one patient harboring A. xylosoxidans. These results illustrate that our findings are robust despite the possible misclassification of A. xylosoxidans and possible variations in test performance of the two culture methods between patients.
Although enrichment in TSB has clearly improved the detection of ESBLs, we believe that there is room for more improvement. We welcome suggestions for further developments.
