ABSTRACT
Invasive candidiasis (IC) is a significant cause of morbidity and mortality. Diagnosis relies on culture-based methods, which lack sensitivity and delay diagnosis. We conducted a systematic review assessing the diagnostic accuracy of PCR-based methods to detect Candida spp. directly in blood samples. We searched electronic databases for prospective or retrospective cohort and case-control studies. Two reviewers abstracted data independently. Meta-analysis was performed using a hierarchical logistic regression model. Random-effects metaregression was performed to assess the effects of study methods and infection characteristics on sensitivity or specificity values. We included 54 studies with 4,694 patients, 963 of whom had proven/probable or possible IC. Perfect (100%) sensitivity and specificity for PCR in whole-blood samples was observed when patients with cases had candidemia and controls were healthy people. When PCR was performed to evaluate patients with suspected invasive candidiasis, the pooled sensitivity for the diagnosis of candidemia was 0.95 (confidence interval, 0.88 to 0.98) and the pooled specificity was 0.92 (0.88 to 0.95). A specificity of >90% was maintained in several analyses considering different control groups. The use of whole-blood samples, rRNA, or P450 gene targets and a PCR detection limit of ≤10 CFU/ml were associated with improved test performance. PCR positivity rates among patients with proven or probable IC were 85% (78 to 91%), while blood cultures were positive for 38% (29 to 46%). We conclude that direct PCR using blood samples had good sensitivity and specificity for the diagnosis of IC and offers an attractive method for early diagnosis of specific Candida spp. Its effects on clinical outcomes should be investigated.
INTRODUCTION
Invasive candidiasis (IC) is a serious cause of morbidity and mortality. In hospitals, Candida spp. represent 8 to 9% of all nosocomial bloodstream infections, and the risk is higher among patients in the intensive care unit (ICU) and cancer patients (19, 81). As many as half of the cases are not diagnosed antemortem (12, 21). In North America, non-Candida albicans spp. are currently more prevalent than C. albicans; Candida glabrata and Candida krusei, which are less susceptible to fluconazole, account for 28% of all candidemias (30). Crude mortality rates are uniformly high, ranging from 40 to 54% (3, 59).
The current gold standard for the diagnosis of IC is blood culture, which takes 24 to 48 h to become positive (16). Identification of the specific Candida sp. might take even longer, delaying appropriate antifungal treatment. Studies consistently show that a delay of 12 to 48 h in appropriate antifungal therapy is associated with significantly increased all-cause mortality that is independent of other risk factors for mortality; adjusted odds ratios range from 2.17 to 4.75 (8, 24, 40, 58, 60, 72). Conversely, the use of empirical antifungal treatment for high-risk patients is highly prevalent, leading to increased costs and adverse ecological effects (25). Non-culture-based methods, such as DNA detection by PCR, have been developed in order to assist in the rapid diagnosis of infections, allowing for the initiation of species-oriented therapy as early as 6 h after the onset of sepsis (52). A bedside scoring system has been developed to guide empirical antifungal therapy for patients colonized with Candida spp. (42). In a cohort of colonized nonneutropenic patients staying >7 days in an ICU without antifungal treatment and with a candida score of <3, the rate of IC was 2.3% (confidence interval [CI], 1.1 to 3.5%), making IC highly improbable. However, a maximal candida score of 5 was associated with an incidence of 23.6% (12.4 to 34.9%), making the candida score less accurate for the positive prediction of IC (43).
We performed a systematic review of studies assessing the diagnostic accuracy of direct PCR on blood samples for IC. We attempted to define the sensitivity and specificity of the test through meta-analysis and to search for modifiers affecting test characteristics.
MATERIALS AND METHODS
Inclusion criteria.We included prospective or retrospective cohort and case-control studies assessing the diagnostic accuracy of PCR-based methods for the detection of Candida spp. directly in blood samples. We included studies reporting on true-positive (TP), false-positive (FP), true-negative (TN), and false-negative (FN) results that had both cases (number of TP plus FN results, >0) and controls (number of TN plus FP results, >0). We excluded PCR testing of blood cultures after incubation or after the identification of growth. No restrictions on language, publication status, year of study, or participants' ages were imposed.
The index tests included any PCR-based method used for the identification of Candida spp. to the genus or species level, including standard, nested, real-time, or reverse transcriptase PCR, using single or multiplex assays. All target genes and primers were accepted. The reference standard was based on established criteria for the definition of IC in neutropenic patients (EORTC criteria) (4, 16) and definitions used in recent clinical trials for nonneutropenic patients (39, 64). We defined three levels for TP and two for TN results. TP level I corresponds to candidemia. TP level II corresponds to proven or probable IC, defined for neutropenic patients by host, clinical, and microbiological criteria (16) and for nonneutropenic patients as the isolation of Candida species from blood or from other normally sterile sites in the presence of at least one indicator of infection within 4 days prior to treatment initiation. Acceptable indicators included inflammation at the site of infection, an elevated or subnormal temperature (≥38.6°C or ≤35.5°C on one occasion or ≥37.8°C on two occasions at least 4 h apart), and systolic blood pressure of ≤100 or ≥30 mm Hg below baseline (39, 64). TP level III corresponds to proven, probable, or possible IC, as indicated by host and clinical criteria without microbiological documentation for neutropenic patients (defined preferably according the revised EORTC criteria [16]) and, for nonneutropenic patients, by sepsis responsive to antifungal treatment with one or more of the recognized risk factors for IC, without microbiological documentation. “TN at risk” refers to patients at risk for IC who do not fulfill the criteria for TP level III; “TN healthy” refers to healthy people. Studies were included if they used these or similar criteria or if these definitions could be applied to the data provided. We expected sensitivity to decrease and specificity to increase from TP level I to TP level III when these individuals were compared to control patients left at risk given the case definition. The comparison of TP level I individuals with healthy people should result in the highest (probably perfect) sensitivity and specificity. For the TP level II and III groups, we expected the sensitivity of PCR to be higher than that of blood culture.
Search strategy.We searched the PubMed, LILACS, NLM Gateway, and KoreaMed databases up to July 2009 and conference proceedings (European Congress of Clinical Microbiology and Infectious Diseases [ECCMID], Interscience Conference on Antimicrobial Agents and Chemotherapy [ICAAC]) between 2000 and 2009, using the search phrase (blood OR serum OR plasma OR buffy-coat OR bloodstream OR candidemia OR candidaemia OR fungemia) AND (PCR OR real-time OR RT-PCR OR reverse-transcription OR nested-PCR OR polymerase chain reaction) AND (candidemia OR candida OR candidaemia). In addition, we scanned the references of all studies included and reviews identified by the search.
Data extraction and assessment of the risk of bias.Two reviewers independently applied inclusion criteria and extracted the data from the studies included. Disagreements were resolved by discussion and in consultation with a third reviewer. When more than one index test (e.g., different PCR primers) or reference standard was assessed, we extracted data separately for each test and reference standard. TP and FN rates were computed by considering only Candida spp. that could be detected by the assay. We extracted data on potential covariates affecting test accuracy or the study's results, including the study year; participants' ages, colonization rates among TP and TN cases, and prior antifungal therapy; type of PCR (real time, nested, or conventional) and methods used (DNA extraction protocol, number of cycles, target gene, and in vitro detection limit); and the blood specimen used for PCR (whole blood, serum, or plasma), whether it was used fresh or frozen, its volume, and the sampling frequency. The risk of bias was assessed independently by two reviewers using the QUADAS tool (41, 80), adapted for our review (see Table S1 in the supplemental material). In addition, we recorded the study design (case-control versus cohort design and whether prospective or retrospective). Authors were contacted to complement missing data. The impact of each item on the results was examined individually.
Statistical analysis.Analyses were performed using Stata/IC, version 10.1 (70). We constructed tables of TP, TN, FP, and FN results per study and reference standard, from which sensitivity and specificity with 95% confidence intervals were calculated. Meta-analysis was performed using a hierarchical logistic regression model (metandi). This method generates a hierarchical summary receiver operating characteristic (HSROC) curve from an individual study's sensitivity and specificity. We report the model's summary sensitivity and specificity values, diagnostic odds ratios (DORs), and likelihood ratios (LRs) with 95% confidence intervals (26, 27). A 95% prediction region is shown on the HSROC curves, defining the sensitivity and specificity values within which we may expect the results of a future study to lie. Random-effects metaregression (metareg) was performed to assess the effects of the moderators listed above on sensitivity or specificity values (and their standard errors) separately (29). Associations with an observed significance (P) of <0.1 are reported. Subgroup analysis was based on the results of the metaregression. In addition, we conducted a prespecified-subgroup analysis of studies performed after the year 2000 in order to address recent technology.
RESULTS
The search strategy identified 1,209 citations, of which 71 potentially relevant citations were selected for further evaluation (see Fig. S1 in the supplemental material). Fifty-four studies published between 1993 and 2009 met the inclusion criteria, providing a total of 101 different comparisons of individual index tests and reference standards (1, 2, 5–7, 9–11, 13–15, 17, 18, 20, 22, 23, 28, 31–38, 44–52, 54–57, 61–63, 65–69, 71, 73–79).
There were 16 case-control studies, 36 prospective cohort studies, and 2 retrospective cohort studies, which included 4,694 patients (963 with proven/probable or possible IC) (see Table S2 in the supplemental material). PCR sampling was always performed prospectively. Seventeen studies assessed adults alone; eight assessed children alone; seven assessed both adults and children; and the others did not state a specific age group. Standard PCR was used in 23 studies, nested PCR in 16, and real-time PCR in 16 (1 study used both nested and real-time PCR [36]). Thirteen studies used serum, while all the others used whole-blood samples. Nineteen studies performed PCR on fresh blood samples, while all the others used frozen, stored blood samples. Antifungal therapy prior to sampling was reported in 11 studies and colonization rates among patients without IC in 15. The genes used as targets for PCR were rRNA genes in 42 studies (18S rRNA in 32, 5.8S and 28S rRNA in 8, and other rRNA genes in 2) and cytochrome P450 L1A1 genes in 6 studies. Other, single studies used the SAP, EO3, HSP, ERG11, CHS1, or ACT1 gene. Twenty-four studies in our review referred to the PCR sample-processing time, which ranged from 4 to 12 h, allowing for the reporting of results within 1 working day in all studies.
The study design and QUADAS methodological assessment are shown in Fig. S2 in the supplemental material and are detailed in Tables S3 and S4 in the supplemental material. Methodological variability was noted between the studies with regard to the recruitment of consecutive participants, the timing of PCR in relation to blood cultures, the testing of all participants with reference tests, and the description of the clinical information available at the time PCR was conducted.
TP I individuals (with candidemia) versus TN healthy people.Fifteen studies assessed a group of patients with proven candidemia versus a group of healthy people using a case-control design. All studies showed a specificity of 1. Sensitivity was 1 in 12 studies and ranged from 0.77 to 0.93 in 3 studies (2, 18, 34). Given these results, summary receiving operator characteristic (SROC) analysis was not done, and the number of studies was too small for formal regression analysis. A noticeable difference was that all studies with sensitivities lower than 100% used serum samples, while all but two of the studies with perfect sensitivity used whole-blood samples (Table 1); thus, sensitivity and specificity were 100% in all studies using whole-blood samples. The study with the lowest sensitivity (34) used the CA1/CA2 Candida actin gene as the target gene, while all others used rRNA or P450 genes.
Results of the diagnostic meta-analyses (hierarchical summary receiver operating characteristic curves)a
TP I individuals (with candidemia) versus TN at-risk patients.Forty-nine studies contributed data for the comparison between TP I and TN at-risk patients; of these, 14 were case-control studies and the rest were cohort studies. The summary sensitivity and specificity values for all studies were 0.95 (CI, 0.88 to 0.98) and 0.92 (0.88 to 0.95), respectively (Fig. 1A). Thirty-one studies showed 100% sensitivity. Restriction of the analysis to studies conducted after the year 2000 (27 studies) resulted in similar results with a higher positive LR (15.4 [9 to 26.4]) (see Fig. S4A in the supplemental material).
HROC curves. (A) TP level I individuals (with candidemia) versus TN at-risk patients (49 studies). (B) TP level II individuals (with proven/probable IC) versus TN at-risk patients (17 studies). (C) TP level III individuals (with proven/probable/possible IC) versus TN at-risk patients (20 studies). In each panel, the shaded square marks the summary point. Open circles, study estimates; solid lines, HSROC curves; dashed lines, 95% confidence regions; dotted lines, 95% prediction regions.
The only variables associated with higher sensitivity on metaregression (P, <0.1) were an in vitro detection limit of ≤10 CFU/ml and the use of a PCR procedure other than the commercial SeptiFast kit (multiplex real-time PCR). A subgroup analysis based on these variables in addition to those observed in the previous analysis (use of whole-blood samples and rRNA or P450 target genes) resulted in higher pooled sensitivity (0.98 [CI, 0.85 to 1]) and slightly narrower prediction regions for sensitivity (23 studies) (see Fig. S5 in the supplemental material). Specificity values in individual studies were associated with methodological variables representing the variability in the definition of patients at risk: prospective cohort studies were associated with lower specificity than retrospective cohort studies and case-control studies, and blinding of the reference standard was associated with higher specificity (P, <0.1 for all).
TP II individuals (with proven/probable IC) versus TN at-risk patients.Seventeen studies contributed to the comparison between TP II and TN at-risk patients; of these, 6 were case-control studies and the rest were cohort studies. The summary sensitivity and specificity values were 0.93 (CI, 0.82 to 0.98) and 0.95 (0.87 to 0.98), respectively (Fig. 1B). Studies conducted after the year 2000 showed higher positive LRs (24.3 [11.3 to 52.4]) and a narrower prediction region for specificity (see Fig. S4B in the supplemental material). Sensitivity was associated with the sample type: it was higher with whole-blood samples than with serum. No other significant associations were observed.
TP III individuals (proven/probable/possible IC) versus TN at-risk patients.Twenty studies were included. As expected, the summary sensitivity value for this analysis was lower than those for the analyses described previously (0.73 [CI, 0.58 to 0.83]). Specificity was maintained at a good level, especially among studies conducted after 2000 (0.95 [0.92 to 0.97]), with narrow prediction CIs (Fig. 1C; see also Fig. S4C in the supplemental material).
Studies that performed more than one PCR test per patient were associated with higher specificity than studies where one sample was taken per patient. DNA extraction with commercial QIAamp (Qiagen) kits resulted in higher specificity than methods based on mechanical lysis. The rates of colonization by Candida spp. among control patients in this analysis, where controls were highly unlikely to have invasive candidiasis, were reported only in 6 studies. There was a trend toward lower specificity with higher colonization, but this did not reach statistical significance (see Fig. S6 in the supplemental material).
Sensitivity of blood cultures for IC.Among TP level II (proven/probable) patients, the pooled blood culture positivity rate was 0.38 (CI, 0.29 to 0.46) (10 studies). Among patients with proven/probable or possible IC, the pooled rate was 0.29 (0.24 to 0.39) (16 studies). The respective PCR positivity rates in the same sets of studies were 0.85 (0.78 to 0.91) and 0.67 (0.62 to 0.72).
Comment.Studies assessing patients with candidemia versus healthy controls showed that PCR of whole-blood samples targeting panfungal genes has 100% sensitivity and specificity. When patients with candidemia constituted the case definition and patients with sepsis at risk for IC without candidemia constituted the control group, the sensitivity was 100% in 31/49 studies and the pooled (HSROC) sensitivity was 0.95 (95% confidence interval, 0.88 to 0.98). As expected, sensitivity decreased as the reference standard became less stringent, from proven through probable to possible IC. In the analysis where patients with cases had proven/probable or possible IC and other patients at risk for IC were controls, the pooled (HSROC) sensitivity was 0.73 (0.58 to 0.83). The pooled specificity was above 90% in all analyses. In studies conducted after 2000, positive likelihood ratios were higher than those in older studies, ranging from 14 to 24 in the different analyses, and the negative likelihood ratio for candidemia was 0.05 (0.02 to 0.14). Among patients clinically suspected of probable or possible IC, the positivity rate of PCR in blood was much higher than that of blood culture (85% versus 38% and 67% versus 29%, respectively). These results are compatible with the conclusion that PCR provides higher sensitivity than blood culture for the diagnosis of IC, with a specificity higher than 90%.
We attempted to identify variables affecting diagnostic accuracy other than the case definition and reference standard used. Higher sensitivity was observed with whole-blood (rather than serum) samples, the use of the QIAamp kit for DNA extraction, the use of panfungal rRNA or P450 genes as the target for PCR, fungus/Candida-specific PCR (rather than a multiplex assay), and an in vitro detection limit of ≤10 CFU/ml (P, <0.1 for all). Specificity was not reduced when serial samples for PCR were obtained every few days or weekly; rather, studies using serial sampling showed higher specificity. We observed a trend showing that colonization by a Candida sp. (without IC) might reduce specificity, but too few studies were available to address this point (see Fig. S6 in the supplemental material). Since nearly all studies used stored frozen blood samples, we could not assess whether fresh samples improve sensitivity. Similarly, due to poor reporting, we could not examine the effect of antifungal treatment or prophylaxis. Studies conducted after 2000 showed higher specificity and less heterogeneity, as reflected in the narrower prediction region around the summary specificity points.
Other advantages of PCR included earlier diagnosis and the possibility of monitoring the persistence or resolution of infection. PCR results were reported to precede candidemia or clinical signs of IC in seven studies, with a range of 1 day to 4 weeks. Five studies reported that persistent positive PCR results on blood were associated with mortality. Badiee et al. noted that PCR remained positive for 14 days among patients who were cured, when weekly PCR sampling was performed, and that PCR positivity persisted longer and until death among patients who died (5, 6). The net clinical benefit afforded by PCR testing for IC could not be assessed. Although PCR results were available to clinicians attending the patient in the several studies performing PCR in real time, only one study directed patients' treatment by PCR results, showing that PCR-directed treatment was initiated 3 days (median; range, 0 to 8 days) before the diagnosis of candidemia by blood culture (44). None of the studies reported the effects of PCR on clinical outcomes. Thus, the studies currently available do not allow assessment of the clinical effects of the higher sensitivity and earlier diagnosis achieved with direct PCR for Candida spp. on blood samples.
The main problem confronting the original studies and this meta-analysis was how to assess a test's accuracy when no gold standard for diagnosis exists. Ultimately, the assessment of PCR on blood for the diagnosis of IC will have to rely on an interventional study, preferably a randomized controlled trial, asking whether management directed by PCR testing improves patients' outcomes. The population should consist of patients at high risk for IC. Serial monitoring and diagnostic PCR samples should be analyzed in real time to guide treatment, while control patients should be managed using conventional culture-based diagnosis. If the primary outcome is all-cause mortality, the sample size needed for such a trial would be about 513 patients per arm (α, 0.05; power, 0.8), assuming a 15% rate of IC among patients at risk, 45% mortality with inappropriate empirical treatment, and a relative risk of 0.5 for death with early antifungal treatment (8, 24, 40, 58, 60, 72). This should be a multicenter effort and will be costly. The current data justify such an interventional study.
Based on the evidence compiled in this review, we believe that PCR in blood might play an important role in improving the outcome of patients with IC through earlier and more-sensitive diagnosis. Given the analysis presented, testing of patients with suspected IC by PCR should accompany, but not replace, blood cultures, and serial sampling might be considered for patients at high risk for IC. PCR targeting panfungal DNA elements, with subsequent species identification, should be applied to whole-blood samples. The assay should have an in vitro sensitivity of at least 10 CFU/ml. Since many assays start with panfungal PCR (a ribosomal DNA sequence between 18S and 5.8S), the same assay can be used for early diagnosis of invasive aspergillosis (53) and IC in hematology patients.
In summary, this diagnostic-accuracy review demonstrated that direct PCR in blood samples may have higher sensitivity for the diagnosis of IC than conventional blood cultures, with a specificity of 90%, which is acceptable for clinical practice. Future studies should assess the clinical effects of this test.
ACKNOWLEDGMENTS
We thank the following physicians for translation of foreign-language papers: Agata Schlesinger (Polish), Morad Wattad (German), Masha Zafran, and Maria Raskin (Russian).
T.A. was supported by a research grant (“Optimal antibiotic treatment of moderate to severe infections: integration of PCR technology and medical informatics/artificial intelligence”) from the Rothschild-Caesarea Foundation.
All authors declare no conflicts of interest.
FOOTNOTES
- Received 9 August 2010.
- Returned for modification 15 September 2010.
- Accepted 17 November 2010.
- Accepted manuscript posted online 24 November 2010.
† Supplemental material for this article may be found at http://dx.doi.org/10.1128/JCM.01602-10.
- Copyright © 2011, American Society for Microbiology. All Rights Reserved.
REFERENCES
- 1.↵
- 2.↵
- 3.↵
- 4.↵
- 5.↵
- 6.↵
- 7.↵
- 8.↵
- 9.↵
- 10.↵
- 11.↵
- 12.↵
- 13.↵
- 14.↵
- 15.↵
- 16.↵
- 17.↵
- 18.↵
- 19.↵
- 20.↵
- 21.↵
- 22.↵
- 23.↵
- 24.↵
- 25.↵
- 26.↵
- 27.↵
- 28.↵
- 29.↵
- 30.↵
- 31.↵
- 32.↵
- 33.↵
- 34.↵
- 35.↵
- 36.↵
- 37.↵
- 38.↵
- 39.↵
- 40.↵
- 41.↵
- 42.↵
- 43.↵
- 44.↵
- 45.↵
- 46.↵
- 47.↵
- 48.↵
- 49.↵
- 50.↵
- 51.↵
- 52.↵
- 53.↵
- 54.↵
- 55.↵
- 56.↵
- 57.↵
- 58.↵
- 59.↵
- 60.↵
- 61.↵
- 62.↵
- 63.↵
- 64.↵
- 65.↵
- 66.↵
- 67.↵
- 68.↵
- 69.↵
- 70.↵
- 71.↵
- 72.↵
- 73.↵
- 74.↵
- 75.↵
- 76.↵
- 77.↵
- 78.↵
- 79.↵
- 80.↵
- 81.↵
