Previous Article | Next Article ![]()
Journal of Clinical Microbiology, May 2005, p. 2188-2193, Vol. 43, No. 5
0095-1137/05/$08.00+0 doi:10.1128/JCM.43.5.2188-2193.2005
Copyright © 2005, American Society for Microbiology. All Rights Reserved.
and
Ajit P. Limaye1,2
Department of Laboratory Medicine,1 Department of Internal Medicine, University of Washington School of Medicine, Seattle, Washington2
Received 29 September 2004/ Returned for modification 13 November 2004/ Accepted 31 December 2004
|
|
|---|
|
|
|---|
There are two obstacles to studying laboratory-related adverse events, which must be overcome to carry out quality improvement projects aimed at reducing these events. The first obstacle is that most laboratories do not collect patient outcome data related to laboratory errors. These data are needed to understand the clinical consequences of laboratory error and to prioritize error reduction measures. The second is the practical challenge of detecting the relatively few cases of laboratory-related adverse events among the large number of tests performed. This obstacle is largely due to the lack of effective, rapid screening methods that laboratory personnel can apply to potential cases.
Sources of potential cases of laboratory-related adverse events include internal laboratory incident reports, risk management incident reports, physician complaints and other forms of physician collaboration, and a variety of daily information system reports. Daily information system reports can be automatically generated and highlight specific areas of suboptimal laboratory practice. Examples of information system reports include turnaround time reports which highlight test results associated with poor turnaround times and the daily list of corrected laboratory reports. A corrected laboratory report consists of an erroneous test result and the correction that was required to that result after the result's initial release. Each of these sources of cases has advantages and disadvantages for detecting laboratory-related adverse events. Although no single source can capture all adverse events, any of the sources can be used for quality improvement projects intended to improve patient safety. Some sources of cases also generate an unmanageably high number of cases and therefore require effective screening methods to significantly enrich for cases that cause adverse events.
In this report we describe a three-stage strategy that was applied prospectively to corrected laboratory reports to identify and characterize the clinical impact associated with problems in clinical microbiology testing. The results indicate that the strategy is feasible to implement and reproducible and detects clinically significant, preventable problems in clinical microbiology testing that are amenable to interventions.
|
|
|---|
Corrected reports. In this prospective study, corrected reports were used as the source of cases of potential laboratory-related adverse events. In the laboratory information system (LIS) used by the participating laboratories, corrected reports were generated in the manual entry system of the LIS by pulling up the incorrect report by accession number, using a special prefix typed in before the accession number. The corrected report was then entered. Each corrected report in the LIS consisted of a description of the result as originally reported with its associated date and time and then the correction to that result with its associated date and time. The LIS had a database search function that allowed flagging of corrected reports, and a list of these corrected reports was automatically generated each morning. The list of corrected reports generated on Monday to Friday was reviewed the same day the list was generated. The reports created on Saturday and Sunday were reviewed on Monday. Routine updates of microbiology results prior to finalization were not considered corrections and were excluded from the study. Examples of routine updates include blood culture daily updates or specific final organism identification that was consistent with the preliminary result.
Evaluating the corrected reports, stage 1. We used a sequential, three-stage method to determine the presence or absence of clinical impact (Fig. 1). The first stage consisted of a rapid, daily review of corrected laboratory reports by a physician (A.P.L. or J.S.) specializing in infectious disease (ID) to identify those cases representing potentially clinically significant errors. This stage ruled out most of the corrected reports for further investigation since most were trivial modifications of unlikely clinical significance. General inclusion criteria for significant corrections included (i) removal or addition of organisms from Gram stain or culture results on a specimen from a sterile site, or pathogenic organisms that are not part of the normal flora on a specimen from a nonsterile site, and (ii) misidentification of organisms by Gram stain, culture gross morphology, biochemical testing, or molecular testing that could cause inappropriate antimicrobial therapy. Corrections of errors with low potential clinical impact were excluded, including (i) trivial changes in a report that do not affect the content of the laboratory result, such as minor typographical corrections; (ii) addition or removal of a nonpathogenic organism in the report that is an obvious contaminant from the site or strictly part of the normal flora; and (iii) misidentification of organisms by Gram stain, culture gross morphology, biochemical testing, or molecular testing that would not normally cause inappropriate antibiotic coverage and treatment.
![]() View larger version (21K): [in a new window] |
FIG. 1. Three-stage method to determine the clinical impact associated with corrected reports. See text for details.
|
Evaluating the corrected reports, stage 3. Cases involving errors that were potentially clinically significant and where clinical impact could not be excluded by record review were selected for the third stage of the method, which was a detailed investigation to determine the patient outcome (Fig. 1). This investigation was performed by one of us (S.Y.) and involved additional review of medical records when necessary and a prompt (within 48 to 72 h) phone interview of at least one clinician directly involved in the care of the affected patient. Specifically, clinicians were asked to assess the presence or absence of the adverse clinical impacts listed previously. Occasionally, the clinicians were contacted again 1 to 2 weeks later to assess the longer-term clinical outcome. This was done in cases when the clinic impact could not be completely assessed at the time of the interview. For example, in a few cases, additional time and data were needed (e.g., laboratory tests, radiologic exams, follow-up physical exams) by the clinician to accurately assess the clinical impact of the error.
When an undesirable clinical impact related to a laboratory error was confirmed, the responsible laboratory was notified of the clinical consequences of the initial error, and the specific cause and nature of the error were further classified using a slightly modified version of a recently published classification scheme (1). The purpose of this classification scheme is to help choose interventions that will reduce the likelihood of the future occurrence of the error. Briefly, the cases were classified by specific clinical impact, the responsibility for the error (laboratory, nonlaboratory, or both), phase of testing (preanalytic, analytic, or postanalytic), and preventability. To facilitate the choice of effective intervention strategies, preventable errors were further categorized based on a cognitive psychology model as a cognitive error (also known as a mistake), which occurs due to lack of knowledge or poor judgment or a noncognitive error (also known as a slip), which occurs due to interruptions in a relatively automatic process (11, 12).
Intra- and interrater reproducibility. An ID physician (A.P.L.) performed the stage 1 evaluation of the corrected reports for about 90% of the corrected reports for this study. For the remainder of the reports, stage 1 evaluation was performed by a second ID physician (J.S.). Both physicians were blinded to the identity of the patient and the laboratory originating the report. To assess the specificity of the stage 1 review of the corrected report by the ID physician, 30 cases that were scored as negative after stage 1 were investigated for patient outcome using interviews of the involved care providers. To assess interrater reliability for stage 1 evaluation, 73 (15.2%) of the 480 corrected reports were chosen randomly and evaluated by both ID physicians. To assess intrarater reliability for the stage 1 evaluation, one of the physicians reevaluated 100 randomly chosen cases at a later date along with new cases, and the second physician reevaluated 94 cases. Inter- and intrarater reliability was measured using the Cohen kappa statistic (6).
|
|
|---|
|
View this table: [in a new window] |
TABLE 1. Results of applying the three-stage method to 480 cases of corrected reportsa
|
|
View this table: [in a new window] |
TABLE 2. Six cases illustrating the use of the three-stage method for evaluating corrected reports
|
Of the 30 cases that were scored by either reviewer as negative for clinical impact after stage 1 evaluation, a clinician interview showed that none resulted in a clinical impact. This confirmed that stage 1 evaluation was highly specific. In contrast, 32 (17.9%) of the 179 cases that were scored as positive after stage 1 led to an undesirable clinical impact. The difference in clinical impact between cases that were scored as negative for potential clinical impact in stage 1 and those that were scored as positive was statistically significant (P = 0.01, chi-square test).
Inter- and intrarater reliability.
There was moderate interrater reliability regarding the results of stage 1 evaluation of the corrected reports by the two reviewers (
= 0.50) and substantial intrarater reliability (
= 0.79 and 0.83, respectively, for the two reviewers).
Classification of laboratory-related adverse events. (i) Summary. Table 3 summarizes the classification of the 32 cases in which there was an adverse clinical impact. The cases were classified according to the specific impact, responsibility for the error, the phase of laboratory testing involved, preventability, and error type if the error was preventable. Most errors were associated with delayed, inappropriate, or unnecessary therapy. The majority of errors were preventable, cognitive, analytic errors that occurred inside the laboratory, and thus were likely to be amenable to laboratory-based interventions, such as increased education or supervision.
|
View this table: [in a new window] |
TABLE 3. Classification of 32 laboratory-related adverse events by specific clinical impact, responsibility for error, phase of laboratory testing involved, preventability, and error type if the error was preventable
|
(iii) Responsibility. Of the 32 laboratory-related adverse events, the laboratory bore all responsibility for the error in 28 (87.5%) of the cases and at least some responsibility in four (12.5%) of the cases. In four (12.5%) cases, nonlaboratory factors also played a role; for example, in one case a physician failed to act appropriately even after being notified of the correct result.
(iv) Phase of testing. Thirty-one (96.9%) of the 32 cases of adverse events involved an analytic error. None (0%) of 32 cases involved preanalytic errors, and one (3.1%) case was due to a postanalytic error, which was a data entry error. Of the 31 cases of analytic errors, 20 (64.5%) involved problems with Gram stain interpretation, 6 (19.4%) involved colony morphology evaluation, 7 (22.6%) related to biochemical testing, 1 (3.1%) was a misinterpretation of a trichrome-stained slide for ovum and parasite examination, and 1 (3.1%) involved in vitro antibiotic susceptibility testing. In 6 (19.4%) of the 31 analytic errors, there was failure to follow an established policy or procedure for the appropriate workup of specimens.
(v) Preventability and error type. Of the 32 cases with adverse clinical impact, 4 (12.5%) cases were considered unpreventable due to highly atypical Gram stain patterns that resulted in the initial misinterpretation by experienced technologists, 26 (81.3%) cases were preventable cognitive errors due to lack of knowledge, and 2 (6.3%) were noncognitive errors. One of the noncognitive errors was the postanalytic data entry error, and the second case was a slide of a body fluid that was inadvertently read before it was stained, causing yeast forms to be missed.
|
|
|---|
The use of corrected reports as a source of adverse events has a number of advantages. First, the list of corrected reports can be generated automatically by the laboratory information system, and therefore dependence on voluntary reporting of errors is avoided. In addition, the list of corrected reports can reflect events occurring within the last 24 h, allowing timely investigation. In contrast, incident reports, which are commonly used to detect laboratory errors (1), require voluntary reporting and usually have significant delays which limit the completeness and accuracy of investigation (15).
Corrected reports have a number of limitations. First, they miss many potential cases of laboratory-related adverse events. For example, delays in laboratory testing are a common source of potential adverse events and most delays do not generate a corrected report since the results are usually correct. Another problem with corrected reports is they are unlikely to randomly sample laboratory-related adverse events. Errors occurring in the laboratory are more likely to be identified and corrected by laboratory staff than errors occurring outside the laboratory, such as mislabeling and suboptimal specimens. In this study, the laboratory was entirely or partially responsible for all 32 cases of adverse events. In addition, the corrected-report method probably is biased toward detecting analytic errors rather than pre- or postanalytic errors. The method also tends to reveal problems with Gram stains, since the results of this screening test are usually reexamined and amended when subsequent culture results appear discrepant. For these reasons, the results obtained should not be extrapolated to estimate the overall rate or type of errors in the laboratory. Complete detection and characterization of laboratory-related adverse events would require the addition of other sources of cases, for example, incident reports, physician complaints, turnaround time reports, etc.
Our results suggest that the first two stages of evaluating the corrected reports, which are relatively rapid, produce a manageable number of cases for the more detailed, time-consuming investigation which takes place in the third stage. Thirty-two percent (Table 1) of all corrected reports were screened positive after the second stage of investigation, meaning that clinical impact had not been ruled out. This is one or two cases per day that would receive the detailed investigation.
In addition, the results indicate that the first stage of the method separates cases with clinical impact from those with no clinical impact. Thus, 32 (17.9%) of the 179 cases that were screened as positive after stage 1 had an adverse clinical impact revealed by a clinician interview. In contrast, when 30 cases screened as negative in stage 1 were investigated in a similar fashion, none had a clinical impact (P = 0.01, chi-square test).
A variety of clinical impacts were attributed to the laboratory errors, most commonly, delayed, inappropriate, or unnecessary therapy (Table 3). In most cases it was difficult to attribute additional clinical morbidity entirely to the laboratory error because of the complex medical conditions of many of the patients. However, increased morbidity due to laboratory error was likely to be present in a significant fraction of these cases. We were able to definitively confirm five cases of increased morbidity, including prolonged sepsis, diarrhea, fever, or dyspnea.
The evaluation of the clinical impact of the laboratory error was not always straightforward. First, there was the difficulty in deciding what the appropriate management would be without the error. For example, one clinician may start antibiotic therapy when new organisms are identified on the corrected urine culture report and state that the therapy was delayed by the error; a second clinician may consider this overtreatment of bacterial colonization. The second physician would not change patient management in this scenario and would consider the error to have no clinical impact. A second methodological problem was that it was not always possible to separate the impact of the laboratory error from the effects of the patient's underlying condition. For example, in one case concerning a patient with an abscess, a laboratory error delayed the appropriate therapy and exacerbated the condition. However, it was not possible to accurately determine the laboratory error's contribution to the patient's morbidity.
Our approach to the problem of obtaining accurate patient outcomes was to defer to the assessment of the physician caring for the patient, as this physician had the most complete information on the patient and was most likely to arrive at the appropriate clinical assessment. This physician was also in the best position to separate the impact of the laboratory error from the effects of the patient's underlying condition. However, in our experience, many clinicians still found this difficult to do and tended to be conservative in their assessment of the impact of the laboratory error. Not surprisingly, it was usually much easier for the clinicians to identify relatively objective impacts such as delayed therapy than impacts such as increased morbidity that require subjective assessment. Therefore, our results likely underestimate some of the clinical consequences of the laboratory errors.
Most of the errors occurred in the analytic phase of testing. In contrast, previous studies of laboratory specialties such as chemistry (7), molecular genetic testing (4), or the stat laboratory (10), as well as previous studies of laboratories in primary care (9) locations or the laboratory as a whole (1, 3, 13) described errors that were most frequently preanalytic. To a certain extent, this difference may reflect the bias of the different methods and sources of cases used. However, the difference may also reflect unique problems in clinical microbiology testing. In clinical microbiology testing, there is heavy reliance on manual work and subjective interpretation, whereas laboratory analyses in sections such as chemistry tend to be largely automated and objective. This point is highlighted by the high number of cases associated with errors in Gram stain interpretation. Gram stains are a manual, subjective, and largely preliminary screening test. The accuracy is affected by many factors, including sampling variability, technical problems, and the expertise of the reader.
Most of the 32 cases of adverse events involved preventable, cognitive errors under the laboratory's authority (Table 3). These cognitive errors are likely to respond to measures aimed at increasing the education, experience, and supervision of laboratory personnel (1, 12). Several such intervention strategies have already been implemented by the laboratories participating in the study.
In summary, the strategy of evaluating selected corrected laboratory reports is effective in identifying preventable adverse events associated with problems in microbiology testing. The errors identified by this method were largely analytic errors of the preventable cognitive type that occurred inside the microbiology laboratory. The errors are likely amenable to laboratory intervention strategies aimed at increasing education and supervision of the laboratory personnel. The methods described here show promise to be applicable throughout the clinical laboratory. Logical directions for future work include determining the portability of the method to other institutions and laboratory divisions (e.g., clinical chemistry, hematology) and modifications to the method to improve its utility. Ideally, the methods would be used in combination with other sources of adverse events, such as incident reports, other management reports (e.g., turnaround time reports), physician complaints, and other forms of collaboration with caregivers.
Present address: Kaiser-Permanente, Vallejo Medical Center, 975 Sereno Drive, Vallejo, CA 94589. ![]()
|
|
|---|
This article has been cited by other articles:
| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Copyright © 2010 by the American Society for Microbiology. For an alternate route to Journals.ASM.org, visit: http://intl-journals.asm.org | More Info»