Previous Article | Next Article ![]()
Journal of Clinical Microbiology, March 2004, p. 1170-1175, Vol. 42, No. 3
0095-1137/04/$08.00+0 DOI: 10.1128/JCM.42.3.1170-1175.2004
Copyright © 2004, American Society for Microbiology. All Rights Reserved.
Evanston Northwestern Healthcare, Evanston,1 Northwestern Memorial Hospital, Northwestern University's Feinberg School of Medicine, Chicago, Illinois,3 Health Outcomes Branch, Division of Healthcare Quality Promotion, Centers for Disease Control and Prevention, Atlanta, Georgia2
Received 28 August 2003/ Returned for modification 22 October 2003/ Accepted 4 November 2003
|
|
|---|
50% increase (compared to baseline) during a three-consecutive-month period. These two methods were compared to standard infection control professional surveillance (ICP) for the detection of clonal outbreaks over 12 months. Overall, a total of seven clonal outbreaks were detected during the 1-year study. Using standard methods, ICP investigated nine suspected outbreaks, four of which were associated with clonal microbes. The 2SD method signaled a suspected outbreak 15 times, of which three were clonal and ICP had detected one. The MI method signaled a suspected outbreak 30 times; four of these were clonal, and ICP had detected one. The sensitivity and specificity values for ICP, 2SD, and MI for detecting clonal outbreaks were 57, 43, and 57% and 17, 83, and 67%, respectively. Statistical methods applied to clinical microbiology laboratory information system data efficiently supplement infection control efforts for outbreak detection. |
|
|---|
Our goal here was to create a monitoring system using microbiologic data that could (i) identify trends in an organism population in a timely manner in order to highlight where infection control intervention might be useful, (ii) be sensitive enough to identify most suspected outbreaks and yet specific so that positive results signaled meaningful problems, and (iii) be easily used in any microbiology laboratory with trained microbiology technologists.
|
|
|---|
Definition of terms.
Two algorithm tools were developed. The first algorithm, 2SD (described in detail below), defined an alert as two standard deviations above a mean. The second, MI (also described in detail below), defined an alert as either a 100% increase from the baseline organism number over 2 months or a
50% increase during a three-consecutive-month period. An "alert" was defined as any time either 2SD or MI signaled the possibility of an outbreak, as indicated by the threshold being exceeded for their predetermined algorithms. To determine the sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) for standard infection control practice, 2SD, and MI surveillance, results were compared to the identification of any clonal outbreak during the 12-month study period. A "clonal outbreak" is defined as a potential outbreak that subsequently was determined to be due to genetically identical microbial pathogens. A "potential outbreak" was an event identified by 2SD, MI or the Infection Control Professionals' (ICP) routine practice and was investigated by molecular DNA fingerprinting analysis of the suspected organisms. A "suspected outbreak" was an event identified by 2SD, MI, or ICP methods that was discussed as a candidate for molecular DNA fingerprinting analysis. Therefore, for the computerized methods, a total of 300 suspected outbreaks may have occurred (25 organisms times 12 months). There was no limitation on the number of suspected outbreaks that could have been identified by the ICP method. We defined an outbreak as an increase in the rate of nosocomial infection above that noted in the past (24). Since the spread of a clonal organism from one patient to another is an undesired goal, the definition of an outbreak was minimally two patients on the same nursing unit or related nursing units with a nosocomial infection due to a single clonal microbial strain.
Statistical analysis. Sensitivity was determined by the number of clonal outbreaks detected by all three surveillance approaches; there were seven such outbreaks in all that formed the performance standard for outbreak detection. Specificity was determined by the performance of the three approaches in detecting the 7 clonal outbreaks within the group of 13 total potential outbreaks throughout the year that were investigated by genetic typing.
Creating the database. In this hypothesis-based computer model, we selected our 25 most common hospital pathogens, purposefully including drug-resistant bacteria, for surveillance (Table 1). The data were available for these organisms in the hospital laboratory information system (Sunquest, Tucson, Ariz.) from 1991 to 1998. The isolate totals for each month of every year were derived from computer generated MIC susceptibility reports, with duplicates removed by a defined algorithm. A duplicate is defined by Sunquest as the same organism from the same patient from the same source in the same month with the identical susceptibility pattern. Slight alterations in the susceptibility pattern of an isolate will result in counting some organisms in replicate. Thus, most but not all replicates were removed. Organism totals included isolates from all inpatient units and outpatient sites. The monthly totals for each organism were entered into an Excel (Microsoft Corp., Redmond, Wash.) spreadsheet for data analysis.
|
View this table: [in a new window] |
TABLE 1. Organisms
selected for use in the database for the 2SD and MI algorithm
tools
|
|
View this table: [in a new window] |
TABLE 2. Example of 2SD: two-factor without replication using mean Citrobacter freundii organisms per month for 8 yearsa
|
50% total increase occurred over the course of a 3-month period, the organism increase triggered a suspected outbreak alert. An example of this is in Fig. 1.
![]() View larger version (46K): [in a new window] |
FIG. 1. Number of positive cultures per month with Acinetobacter baumannii complex analyzed using the MI algorithm (January 1997 to December 1999).
|
Infection control initiated investigation. Practices to determine whether hospital-associated infections may be occurring were based on current Centers for Disease Control and Prevention (CDC) recommendations for definition of a nosocomial infection (5). Nosocomial infections were detected by ongoing surveillance using trained nurses for intensive care units (ICUs), including a neonatal ICU, and postsurgery units applying standard nosocomial infection definitions. Additional data were collected through manual review of microbiology reports and patients' medical records, direct observation of medical and nursing practice, and active surveillance using rectal cultures of patients residing on nursing units caring for those with high-risk conditions. In addition, there was an evaluation of suspected nosocomial infections reported by healthcare providers throughout the hospital.
Time analysis. For each trended organism that exceeded a 2SD or MI threshold, additional computer generated reports were done to determine whether there were any nursing unit trends, and if these appeared evident, the patient admission histories were reviewed to determine whether the positive cultures had been obtained more than 2 days after admission. If this revealed a high number of nosocomial positives, the potential for drug resistance or specimen source, then the trends were evaluated.
The use of microbiology laboratory data for development and validation of a tracking tool to enhance detection of nosocomial infections was approved by the Institutional Review Board of Northwestern University and the CDC.
|
|
|---|
|
View this table: [in a new window] |
TABLE 3. Suspected outbreaks detected by the 2SD and MI algorithm tools during 1999
|
The sensitivity, specificity, PPV, and NPV for the detection of clonal outbreaks by the ICP, 2SD, and MI methods involving the 13 potential outbreaks during 1999 are compared in Table 4.
|
View this table: [in a new window] |
TABLE 4. Sensitivity, specificity, PPV, NPV for ICP, 2SD, and MI for detecting a clonal outbreak
|
In November and December 1999, MI and 2SD analyses (2SD gave an alert only in December) again signaled a suspected outbreak for both clinical and surveillance isolates of VREF. Based upon patient room assignments, three nursing units had multiple patients with VREF in November and five nursing units had multiple patients with VREF in December. Typing of these isolates revealed that three of the suspect nursing units each had three to five patients with a common genomic strain type in the course of 1 month. Importantly, each unit had three to four patients with the identical VREF clone that were positive for the first time, indicating the high likelihood that a nosocomial outbreak of VREF had been detected.
The greatest expenditure of time was the initial creation of the database. The ongoing process of retrieving statistics electronically from the Sunquest susceptibility (MIC) reports required approximately 2 h of operator time per month, plus the time needed to assess each suspected outbreak to determine whether further investigation was needed. Overall, these trend analyses used in our report can be done over the course of one 8-h working day each month, provided that the organism numbers are easily gathered from the laboratory information system using minimal operator time. During the period of the present study, there were three full-time infection control professionals assigned to the inpatient services at NMH.
|
|
|---|
Selecting a "gold standard" for a comparison such as this is difficult because any system may miss at least some outbreaks. We chose to compare detection of clonal events to provide an objective measure of performance between the varied approaches, even though no single one may be optimal. Support to the findings of our investigation comes from the recent report by Poulakou et al., who compared analysis of microbiology information system data to two other computerized surveillance approaches and to a reference standard in which each patient in the general and cardiac surgery department was monitored for surgical-site infection (G. Poulakou, A. Chalfine, A. Ben Ali, D. Cauet, J. Gonot, F. Goldstein, and J. Carlet, Prog. Abstr. 5th Eur. Cong. Chemother. Infect., abstr. FP1.01, 2003). These researchers found that monitoring microbiology laboratory data performed the best, with a sensitivity exceeding 80%, for the detection of these infections (Poulakou et al., Prog. Abstr. 5th Eur. Cong. Chemother. Infect.). Our results suggest that the current practice of focused infection control surveillance to detect nosocomial infections can be supplemented through the use of data generated in the clinical microbiology laboratory, even when relatively rudimentary computer tools based on threshold analysis are utilized. The 2SD and, particularly, the MI methods of analysis using monthly microbiologic data have potential use as an adjunct to routine outbreak detection. These methods met most of our objectives for enhancing infection control activity by identifying trends in a timely manner, while demonstrating reasonable sensitivity and specificity. The results are comparable to the 84% sensitivity and 48% specificity suggested by Laxson et al. in 1984 when they used positive cultures from the microbiology laboratory to detect possible nosocomial infections (11). One year later Schifman and Palmer demonstrated that by using excess rates of positive cultures, on a location-specific basis, far out-performed traditional surveillance in detecting small clusters of cross-infections (20). More recently, Glenister et al. used manual review of microbiology laboratory results to detect hospital-acquired infections and found a sensitivity similar to ours, depending on whether laboratory test results were only reviewed (48%) or if there was a twice-weekly follow-up visit to the nursing unit for a discussion of potential infections (71%). These surveillance approaches required only one-sixth to one-third the amount of time as their standard surveillance method (8).
One important aspect of the computer tools we describe here is that they can be used without the need for more resources in the microbiology laboratory, other than the time for data retrieval and trend analysis that was approximately 1 day per month for our healthcare organization. Significantly, not all clonal events during the study period were detected with routine infection control surveillance methods, indicating that there is potential for using the microbiology information system to identify nosocomial outbreaks of infection that currently go undetected using current standard practices. Our concurrent evaluation of 2SD and MI methods, along with routine infection control surveillance, found four suspected outbreaks that were not detected by standard practice methods, and three of these events proved to involve clonal isolates. Furthermore, 2SD analysis detected 11, and MI analysis detected 24 suspected outbreaks that were never investigated.
Interestingly, the MI method identified one clonal outbreak a month sooner than the 2SD approach (VREF in November), which reinforces the remainder of our data suggesting that MI is the more sensitive of the tools, with similar specificity. Another benefit of MI is that organism trends are noted as the numbers are increasing and a developing problem can be seen over a period of 2 to 3 months, even before the number meets the 2SD upper boundary to signal a problem. The possible disadvantage of MI is that it signaled twice as many suspected outbreaks as 2SD (including one proven nonclonal event for which 2SD did not generate an alert), indicating potentially too much monthly variation in recovered organisms to make MI the optimal surveillance tool.
We desired a method that was sensitive enough to identify all suspected outbreaks while sufficiently specific so that a suspected outbreak is likely to represent a true nosocomial infection event. When suspected outbreaks identified by the 2SD method were compared to the clonal outbreaks investigated by the ICP approach, 2SD analysis did not signal four of the seven outbreaks that were eventually detected and thus detected 43% of the clonal outbreaks. The MI tool performed better than the 2SD method and detected four of the seven clonal outbreaks (57%) found during 1999, which was equivalent to standard infection control practice performance. This relative lack of sensitivity for the current computer tools may be due to the fact that all of the clonal outbreaks investigated during this 1999 study consisted of small numbers (three to five) of patients. Databases using hospital-wide figures may mask small outbreaks that occur on individual units. This limitation is highlighted by the report of Price et al., who detected an outbreak of bacteremia in an outpatient dialysis unit using the 2SD and MI approaches (18). Although these algorithms did recognize the potential problem that had not been detected by routine infection control surveillance, detection required some 8 months and only occurred when the bacteremias in a given month were primarily associated with a single bacterial genus (Enterococcus sp.). The sensitivity of computerized analysis may increase if unit specific databases for detecting outbreaks were created. If unit specific trends were to be monitored, the organisms chosen for the database could be selected based on the population of patients on the unit surveyed and include important drug-resistant organisms so as to increase outbreak detection sensitivity.
In conclusion, an active, integrated infection control program has a major positive medical and economic impact on a healthcare organization (10). Our data demonstrates that a fully manual surveillance approach does not capture all of the potential outbreaks in an active medical center practice. Over 25 years ago it was recognized that computer tools could help the hospital epidemiologist and infection control committee in managing healthcare-associated infectious diseases (9). Newer reports have confirmed that hypothesis (1). We found that monitoring the activity of 25 important nosocomial pathogens with the 2SD and MI methods did enhance our infection control program by detecting four potential outbreaks not investigated by the Infection Control and Prevention Department in 1999, three of which were due to the unsuspected spread of clonal organisms. The amount of additional work required for use of these two tools totaled approximately 8 h per month. New computerized tools for increased detection of potential infection control opportunities are rapidly being developed (3) and offer great potential for system-wide surveillance in detecting healthcare-associated infectious diseases (16). After this investigation the 2SD and MI tools were routinely used in our infection control program and continued to supplement outbreak detection at our medical center (18). We have recently adopted a new computer tool with considerably enhanced analysis capacity and over the next 12 months will compare the data mining system to the 2SD and MI analysis methods (16). Use of statistical methods and information technology to analyze microbiology culture result data can supplement traditional infection control outbreak detection in an efficient and cost-effective manner to enhance infection control practice.
|
|
|---|
This article has been cited by other articles:
| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Copyright © 2009 by the American Society for Microbiology. For an alternate route to Journals.ASM.org, visit: http://intl-journals.asm.org | More Info»