Previous Article | Next Article ![]()
Journal of Clinical Microbiology, September 2008, p. 2924-2929, Vol. 46, No. 9
0095-1137/08/$08.00+0 doi:10.1128/JCM.00623-08
Copyright © 2008, American Society for Microbiology. All Rights Reserved.

Department of Pharmaceutical and Nutrition Care, The Nebraska Medical Center, Omaha, Nebraska 68198,1 Department of Pharmacy Practice, College of Pharmacy, University of Nebraska Medical Center,2 Section of Infectious Diseases, Department of Internal Medicine, College of Medicine, University of Nebraska Medical Center,3 Department of Pathology and Microbiology, College of Medicine, University of Nebraska Medical Center, Omaha, Nebraska 681984
Received 1 April 2008/ Accepted 1 July 2008
|
|
|---|
|
|
|---|
The ability to rapidly distinguish between the presence of S. aureus and CoNS in a blood culture is essential for optimal patient management. Both patient outcomes and the costs associated with treatment are affected by the method used for identification. Unnecessary or inappropriate empirical therapy may be eliminated or replaced with targeted therapy. Follow-up cultures, used to discriminate contamination from true infection, may be eliminated, decreasing laboratory costs, or collected sooner, expediting diagnostic accuracy and appropriate treatment. The length of the hospital stay and/or resource utilization may be decreased by not treating CoNS contamination or by diagnosing an S. aureus infection more quickly (1, 5). Therefore, careful evaluation of the available options to differentiate staphylococci is essential in order to select the alternative that generates accurate results in a timely manner at a reasonable cost.
Currently, most clinical microbiology laboratories use either the tube coagulase test or the latex agglutination test to differentiate S. aureus from CoNS. These tests are performed on the overnight culture obtained from the positive blood culture and thus are not interpreted until 24 h after the blood culture becomes positive. During this time, clinicians frequently make empirical therapy decisions because of the possibility that the organism is S. aureus. New tests are available that allow more rapid identification of staphylococci directly from blood. They include the peptide nucleic acid fluorescence in situ hybridization (PNA-FISH; AdvanDx, Woburn, MA) assay, which can be interpreted in as little as 3 hours, and real-time PCR (RT-PCR; Cepheid, Sunnyvale, CA), which can be interpreted in under 1 hour. Both of these tests have improved sensitivity, but at a higher cost, compared to the tube coagulase test interpreted at 24 h (TCT24). Recent data suggest that the tube coagulase test may be performed directly from blood and may also be interpreted at 4 h (TCT4) with little loss of sensitivity or specificity (2).
The impact of variability in the turnaround time, sensitivity, specificity, and acquisition cost on the cost-effectiveness of these tests is currently unknown. Estimating these relationships through decision modeling is an efficient way to quantify the benefit and risk of using any of these tests when data from well-designed prospective trials are not available. The objective of this study was to compare the cost-effectiveness of TCT24, PNA-FISH, RT-PCR, and TCT4 for differentiation of staphylococci directly from blood culture bottles when one of two blood cultures yields gram-positive cocci in clusters.
|
|
|---|
Diagnostic strategies. The diagnostic strategies included various methods used for detection of S. aureus directly from blood culture bottles. Data suggest that the tube coagulase test can be performed directly from blood culture bottles and interpreted earlier than the standard 24-hour turnaround time (2). The methodologies for both TCT24 and TCT4 were the same, with the exception of the turnaround time. More specifically, several drops of the positive blood culture broth were added to a tube containing 0.5 ml of rabbit plasma with EDTA (BBL, Cockeysville, MD). The tubes were incubated at 35°C for either 4 or 24 h for TCT4 or TCT24, respectively. Any clotting was interpreted as indicative of S. aureus. The time required for the laboratory technician to perform the test and read and report the results was approximately 5 minutes per specimen.
The PNA-FISH assay involved a number of steps (AdvanDx, Woburn, MA). First, a smear was prepared on a microscope slide using a drop of the positive blood culture broth and a fixation solution. Hybridization was then performed by adding S. aureus PNA to the microscope slide and incubating the slide at 55°C for 90 min. The slide was subsequently washed with a preheated, prepared wash solution and incubated at 55°C for an additional 30 min. Finally, one drop of mounting medium was added to the smear prior to the slide being observed using a fluorescence microscope. S. aureus was identified as multiple bright-green fluorescent clusters of cocci. We estimated that the time required by the laboratory technician for preparation of the test and to read and report the results for a single specimen was approximately 50 min, with 5 min added for each additional specimen within the batch.
The RT-PCR test was performed by inoculating the provided cartridge with the positive blood culture broth and inserting the cartridge into the required instrument (Cepheid, Sunnyvale, CA). Four reactions occurred simultaneously within a single cartridge. The reactions targeted S. aureus protein A (Spa), the mecA gene, a proprietary sequence of the SCC mec cassette, and the S. aureus-specific orfX gene and included an internal control for determination of the presence of S. aureus and methicillin-resistant S. aureus. We estimated that the time required by the laboratory technician for preparation of the test and reading and reporting the results was approximately 5 min per specimen.
Probability estimates. Probability and cost estimates for the base case and sensitivity analysis came from the literature and institutional data. Our institution is a 689-bed academic medical center with three adult intensive-care units, a pediatric intensive-care unit, a neonatal intensive-care unit, and large hematopoetic stem cell and solid-organ transplant populations. Using sensitivity analysis, the probabilities and costs were varied over a range of plausible values to assess the impact of each variable on the final conclusion. Sensitivity analysis allows one to see if changing the probability or cost of an event changes the conclusion drawn in the base case. If the conclusion does not change, the model is considered robust, generating enhanced confidence in the model's usefulness for decision making. If the conclusion changes, then the model is sensitive to changes in the estimates, suggesting that the model may be less useful in different environments as a decision-making tool.
Baseline probabilities.
The decision model and variables used for the various methods of testing are shown in Fig. 1A to D. The first chance node for modeled patients is to evaluate the presence of risk factors for harboring true infection when one of two blood cultures yields CoNS. The presence of any of the following risk factors would cause increased suspicion of true infection due to CoNS rather than contamination: (i) a white blood cell count of <2,000 cells/mm3 or >12,000 cells/mm3 plus >10% bands and a body temperature of <36°C or
38.5°C or systolic blood pressure of <90 mm Hg, (ii) a neonate, or (iii) the presence of a central line or other indwelling intravascular device (e.g., a pacemaker or vascular graft). The probability of the presence of these risk factors was determined from a review of a convenience sample of 182 patients with one of two blood cultures yielding either CoNS or S. aureus at our institution.
![]() ![]() View larger version (70K): [in a new window] |
FIG. 1. Decision model and variables for TCT24, TCT4, PNA-FISH, and RT-PCR. p, probability; c, cost; Coag +, coagulase-positive staphylococci (S. aureus); Coag –, CoNS; RiskPos, presence of risk factors for true infection plus coagulase-positive staphylococci; NoRiskPos, lack of risk factors for true infection plus coagulase-positive staphylococci; TruePos, true coagulase positive; TrueNeg, true coagulase negative; FalsePosCont, false coagulase positive plus contamination; TrueNegCont, true coagulase negative plus contamination; FalseNegCont, false coagulase negative plus contamination; Vanco, vancomycin; BloodCult, blood culture. (A) Decision model and variables for TCT24. (B) Decision model and variables for TCT4. (C) Decision model and variables for PNA-FISH. (D) Decision model and variables for RT-PCR.
|
The third chance node reflects the probability that a test result will be either a true positive or a true negative (Table 1), which was calculated from the sensitivity and specificity of the diagnostic tests and the underlying institutional rate of S. aureus presence when one of two blood cultures is positive by Gram staining for staphylococci. The sensitivities and specificities of TCT24 (90% and 100%, respectively) and TCT4 (85% and 100%, respectively), directly from blood culture bottles, were derived by our microbiology laboratory by testing consecutive blood cultures initially characterized by Gram staining as yielding staphylococci (personal communication). The sensitivity and specificity of the PNA-FISH assay (98.5% and 98.5%, respectively) were taken from the manufacturer's data (S. aureus PNA FISH Staphylococcus aureus culture identification kit package insert; AdvanDx, Woburn, MA). The sensitivity and specificity of the RT-PCR test (99.7% and 99.1%, respectively) were taken from the manufacturer's data for a commercially available kit (BD GeneOhm StaphSR assay package insert; BD Diagnostics, Quebec, Canada), although the other modeled RT-PCR variables are associated with an investigational kit (Cepheid, Sunnyvale, CA).
|
View this table: [in a new window] |
TABLE 1. Model variable values and sensitivity ranges
|
Baseline costs. Cost data are presented from the institutional perspective and are expressed in U.S. dollars. The costs of vancomycin ($15/g), blood cultures ($29.50/culture and identification), TCT24 ($3.02/test), and TCT4 ($3.02/test) were based on the costs to our institution, including supplies and labor. The cost for PNA-FISH ($66.06/test) is based on batching the test runs three times daily with eight tests per batch, including supplies and labor but excluding the $4,500 start-up cost for instrumentation. The cost for RT-PCR ($45/test) is according to the manufacturer and includes supplies and labor but excludes $135,000 in start-up costs for instrumentation. All costs were varied through a range of values using sensitivity analysis (Table 2).
|
View this table: [in a new window] |
TABLE 2. Costs of tests
|
|
|
|---|
Model results. In the base case, TCT4 and TCT24 ($78 per patient) were equally cost-effective and more cost-effective than RT-PCR and PNA-FISH ($120 versus $165 per patient, respectively) for the identification of staphylococci directly from blood culture bottles.
Sensitivity analyses. One-way sensitivity analyses were conducted on all variables. Three variables accounted for 95.5% of the model's variability: cost of blood culture, cost of vancomycin, and probability of contamination when CoNS was identified in a blood culture. A series of two-way sensitivity analyses were conducted using these variables. Regardless of the values used for these variables, the cost-effectiveness relationship between the tests was unchanged. Thus, the advantages of TCT4 and TCT24 were robust.
A series of threshold analyses were also conducted to determine the level at which either RT-PCR or PNA-FISH would become a more cost-effective alternative to TCT4 or TCT24. Holding all other variables at base case levels, the RT-PCR test would have to cost less than $9.81/test in order to be the most cost-effective alternative. Similarly, the cost per test for PNA-FISH would have to be less than $2.61.
|
|
|---|
While sensitivity and specificity are generally important comparative attributes of diagnostic tests, the variation in sensitivity and specificity among TCT24, PNA-FISH, RT-PCR, and TCT4 had a minimal effect on our model. The practice of subculturing Gram stain-positive blood cultures to agar plates strongly affects the impact of test sensitivity and specificity on the model. Access to these plates allows definitive identification of the organism (via latex agglutination) 24 h after the culture becomes positive, preventing organism misidentification beyond 24 h and significantly decreasing the probability of a missed infection. Plate subculturing, therefore, increases the acceptability of the lower sensitivity achieved with TCT24 and TCT4 and may obviate the need for the higher sensitivity and associated expense of using PNA-FISH and RT-PCR. Furthermore, because the presence of CoNS can be defined as infection or contamination within 48 h based on follow-up blood cultures, the relevancy of the sensitivity and specificity of diagnostic tests declines beyond this time point.
With an increased demand for more rapid reporting of results in the laboratory, a shorter turnaround time for PNA-FISH and RT-PCR appears attractive (6, 7, 12, 16). The PNA-FISH assay can be interpreted at 3 hours, a significant difference from TCT24. However, this time saving may be more theoretical than practical. For instance, the testing of individual samples at the time the blood culture bottle is determined to be Gram stain positive for staphylococci, in order to derive the greatest benefit from the attenuated interpretation time, may not be feasible for most institutions due to workflow issues. Rather, in most laboratories, samples would be batched to increase efficiencies necessitated by manpower and labor and supply cost considerations. Batches may be run as little as one to three times a day. The value of a short time to interpretation, when its benefit cannot be fully realized, renders the PNA-FISH assay less cost-effective.
We modeled an investigational RT-PCR kit (Cepheid, Sunnyvale, CA) because of the rapid generation of results (within 1 hour) and ease of use in contrast to the commercially available RT-PCR kit (BD GeneOhm StaphSR assay; BD Diagnostics, Quebec, Canada), which would be batched similarly to PNA-FISH due to labor intensity. Thus, unlike PNA-FISH, RT-PCR may feasibly be performed on individual samples, allowing rapid species identification and determination of the methicillin susceptibility of the isolate. However, similar to PNA-FISH, this time-saving benefit may be more theoretical than actual. Obtaining bacterial identification sooner may result in the administration of fewer unnecessary doses of vancomycin, assuming that physicians will wait 2 hours for the results of testing before starting empirical therapy. However, in the model, this scenario occurs only when patients who have no risk factors for true infection have a blood culture contaminated with CoNS. Furthermore, rapid bacterial identification is problematic when it is erroneous. The outcome associated with a misidentification of S. aureus, when bacterial growth is actually due to CoNS contamination in patients with risk factors for true infection, is the administration of unnecessary antimicrobial therapy. Conversely, a potential benefit of RT-PCR is earlier decision making related to patient isolation, which would be made possible by more rapid access to information about methicillin resistance. However, little is understood about how the timing of isolation affects patient outcomes and costs. Another potential benefit of RT-PCR is an earlier change from vancomycin to an antistaphylococcal penicillin when methicillin susceptibility is confirmed because an antistaphylococcal penicillin is considered optimal therapy for methicillin-susceptible S. aureus. However, factors affecting the timing of isolation and a switch to optimal therapy due to the determination of methicillin susceptibility are beyond the scope of this model.
TCT4 provides a more rapid result at a cost equivalent to that of TCT24. Importantly, the choice between TCT4 and TCT24 results in differential administration of vancomycin. Specifically, in contrast to TCT24, TCT4 prevents a missed dose of vancomycin in patients with no risk factors for true CoNS infection but who are actually harboring S. aureus. The impact on patient outcomes resulting from the elimination of a single dose of vancomycin in the first 48 h of detection of possible bacteremia is unknown and is not considered in this model. However, it is reasonable to assume that missing a dose would result in inadequate therapy, which is not optimal for patient outcomes (8-10) Thus, the prevention of a missed dose is a potential advantage of TCT4 over TCT24.
In conclusion, our model highlights two unexpected findings. The variation in sensitivity and specificity and time to interpretation of TCT24, PNA-FISH, RT-PCR, and TCT4 have limited impact on the cost-effectiveness of the four strategies. Thus, the increased sensitivity and rapid turnaround time of the newer tests do not appear to be worth the increased price. Of note, our model does not address the methicillin susceptibility of S. aureus. Future research on the impact of this factor is needed. Based on our current model results, we believe that TCT4 should be investigated further as a cost-effective means of rapidly differentiating staphylococci directly from blood culture bottles.
No financial support was received for this study.
Published ahead of print on 9 July 2008. ![]()
|
|
|---|
| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Copyright © 2009 by the American Society for Microbiology. For an alternate route to Journals.ASM.org, visit: http://intl-journals.asm.org | More Info»