ABSTRACT
Clostridium difficile colonizes the gastrointestinal (GI) tract, resulting in either asymptomatic carriage or a spectrum of diarrheal illness. If clinical suspicion for C. difficile is low, stool samples are often submitted for analysis by multiplex molecular assays capable of detecting multiple GI pathogens, and some institutions do not report this organism due to concerns for high false-positive rates. Since clinical disease correlates with organism burden and molecular assays yield quantitative data, we hypothesized that numerical cutoffs could be utilized to improve the specificity of the Luminex xTAG GI pathogen panel (GPP) for C. difficile infection. Analysis of cotested liquid stool samples (n = 1,105) identified a GPP median fluorescence intensity (MFI) value cutoff of ≥1,200 to be predictive of two-step algorithm (2-SA; 96.4% concordance) and toxin enzyme immunoassay (EIA) positivity. Application of this cutoff to a second cotested data set (n = 1,428) yielded 96.5% concordance. To determine test performance characteristics, concordant results were deemed positive or negative, and discordant results were adjudicated via chart review. Test performance characteristics for the MFI cutoff of ≥150 (standard), MFI cutoff of ≥1,200, and 2-SA were as follows (respectively): concordance, 95, 96, and 97%; sensitivity, 93, 78, and 90%; specificity, 95, 98, and 98%; positive predictive value, 67, 82, and 81%;, and negative predictive value, 99, 98, and 99%. To capture the high sensitivity for organism detection (MFI of ≥150) and high specificity for active infection (MFI of ≥1,200), we developed and applied a reporting algorithm to interpret GPP data from patients (n = 563) with clinician orders only for syndromic panel testing, thus enabling accurate reporting of C. difficile for 95% of samples (514 negative and 5 true positives) irrespective of initial clinical suspicion and without the need for additional testing.
INTRODUCTION
Clostridium difficile causes a spectrum of gastrointestinal (GI) illnesses ranging from mild diarrhea to toxic megacolon and death (1). Asymptomatic carriage has been identified in up to 15% of healthy adults, with increased colonization rates in individuals with repeated exposures to dysbiotic agents (e.g., antibiotics, chemotherapy, immune suppressants, etc.) and in up to 50% of elderly patients residing in long-term care facilities (2). A review of the primary causes of nosocomial diarrhea highlight ≤20% of cases as attributable to C. difficile infection, with the majority due to medications, enteral feeding, or underlying illness (3). This combination renders nucleic acid amplification tests (NAATs) highly susceptible to false positives (4), the consequences of which include subjecting patients to the side effects of unnecessary antibiotics, promotion of multidrug-resistant enteric microbiota (including vancomycin-resistant enterococci), further propagation of microbial dysbiosis, and failure to identify the underlying cause of diarrheal illness (3). In contrast, protein toxins are acid, heat, and enzymatically labile (5–7) and susceptible to neutralization by host-derived (8–10) and therapeutic antibodies (11), contributing in part to the reduced sensitivity of toxin enzyme immunoassays (EIAs) to milder forms of disease harboring low organism burden and toxin production (12). The consequences of false-negative results can be severe, including increased patient morbidity and mortality and loss of clinician confidence in C. difficile testing results.
These innate testing vulnerabilities are well known and in the absence of definitive expert consensus guidelines have sparked three major approaches to directed C. difficile diagnostic testing (1, 5). A March 2016 Clinmicronet survey (a global listserv of doctoral clinical microbiologists) showed the following breakdown in diagnostic testing practices among respondents (n = 70): (i) NAAT-only approach (78%), (ii) an algorithm utilizing glutamate dehydrogenase (GDH) as a screen followed by toxin EIA and adjudication of toxin-negative cases with PCR (13%); two-step algorithm [2-SA], and (iii) an algorithm utilizing GDH or NAATs as a screen and confirmation of toxin-positive cases with a toxin EIA (9%) (5). Regardless of local diagnostic testing practices, typical case presentations such as diarrheal illness in an antibiotic-exposed elderly individual evoke high suspicion for C. difficile, triggering clinicians to order directed testing. In contrast, patients without obvious antibiotic use or recent hospital exposure in the preceding 12 weeks (i.e., community onset) raise a broad differential with low suspicion for C. difficile, prompting orders for testing methodologies (stool culture, ova parasite, and syndromic panels) that do not detect this organism (12, 13). Some laboratories that perform multiplex assays targeting C. difficile choose to hide results for this organism out of concern for high false-positive rates. With an estimated incidence of community-associated C. difficile infections (CA-CDI) ranging from 1.5% to 15% of total CDI (14–17), it is troubling to note that up to 60% do not elicit an order for directed testing (18). Although most of these infections will be self-limited, a subset of patients would likely benefit from treatment enabling faster symptom resolution and prevention of serious sequelae (19). Furthermore, knowledge of C. difficile involvement in their diarrheal illness would relieve patient anxiety, trigger precautions around susceptible close contacts, and increase clinical suspicion should symptom onset recur.
Recent efforts to improve NAAT specificity have honed in on quantitative real-time PCR cycle threshold (CT) values reflective of organism burdens that predict toxin EIA positivity (20–24). Although reported out qualitatively, the Luminex xTAG GI pathogen panel (GPP; targets 14 GI pathogens including C. difficile) also yields quantitative data measured in median fluorescence intensity (MFI) units (13, 25, 26). In this study, we show that an algorithmic approach utilizing high and low quantitative MFI cutoffs improves the specificity of the Luminex GPP for active infection without compromising the assay's high sensitivity for organism detection. This approach enables accurate detection and reporting of C. difficile from this multiplex assay irrespective of clinical suspicion, enabling identification of a subset of previously undiagnosed patients.
MATERIALS AND METHODS
Processing and analysis of patient samples utilizing the 2-SA and the Luminex xTAG GPP.Stool samples with clinician orders for C. difficile testing sent to the Clinical Microbiology and Immunology Laboratories at the University of North Carolina (UNC) Health Care Hospital in Chapel Hill, NC, were assayed utilizing the 2-SA (1). At our institution, the 2-SA begins with the Alere C. diff Quik Check Complete EIA (Waltham, MA) (used per the manufacturer's instructions), which simultaneously tests for the presence of GDH (a sensitive screen targeting C. difficile with cross-reactivity to other Clostridium species) and toxins A and B. All GDH-negative (GDH−) samples are reported out as negative, GDH-positive (GDH+)/toxin-positive samples are reported out as positive, and GDH+/toxin-negative samples are reflexed to the Cepheid Xpert C. difficile PCR assay targeting the toxin B gene (tcdB) (Sunnyvale, CA) (used per the manufacturer's instructions) and reported based on the PCR result. The 2-SA is restricted to specimens that take the shape of the container and is not performed on patients that are <1 year old, have documented laxative use (≤48 h prior), or have had a negative C. difficile test within the previous 7 days or a positive one in the past 14 days.
Our institution additionally offers the Luminex xTAG GI pathogen panel (GPP) multiplex PCR assay (Austin, TX), which is capable of simultaneously detecting 14 GI pathogens (13). Unlike 2-SA testing, there is no age restriction for ordering the GPP. The test is performed on stool specimens from outpatients and inpatients (hospitalized for <3 days) that conform to the shape of the container. The pathogens reported out from this assay at our institution include Campylobacter, Salmonella, Shigella, Escherichia coli O157, Shiga toxin-encoding E. coli (STEC), Giardia, Cryptosporidium, rotavirus, and norovirus. During the time frame of the current study, C. difficile GPP data were not reported out of concern for high false-positive rates. The procedure is performed as per the manufacturer's instructions, with the exception that raw median fluorescence intensity (MFI) values were analyzed for this study. The assay involves PCR amplification and hybridization of biotinylated amplicons to cDNA probes bound to beads with unique fluorescence spectral patterns and to phycoerythrin (PE)-labeled streptavidin. The beads are then passed through a flow cytometer, identified via unique UV light fluorescence patterns, and analyzed for the presence and quantity of bound amplicons (MFI values). The identified bead is matched to a particular organism, and the associated MFI value is compared to a predetermined threshold to determine the presence or absence of the infectious agent. For C. difficile, the standard positive MFI threshold is ≥150 for genes encoding either toxin A or toxin B. Although typically concordant, data from averaged MFI values for toxins A and B were evaluated in the current study to improve assay specificity and mitigate false positives caused by high MFI values associated with one toxin alone. Given the low initial clinical suspicion for C. difficile in samples submitted for syndromic panel testing, we favor the improved specificity of data interpretation with averaged MFI values.
At our institution, GPP analysis of stool samples is performed in singlicate, and the reproducibility of toxin A and toxin B MFI values in this retrospective data set could not be assessed. Based on GPP precision data submitted to the FDA and precision studies performed in our laboratory for targets other than C. difficile (not reported from the GPP for clinical use at the time of this study), we would expect some quantitative but not qualitative variation in MFI values on repeat testing.
Identification of a GPP MFI threshold predictive of 2-SA and toxin EIA positivity.At our institution, clinicians ordering the 2-SA alone have a relatively high suspicion for C. difficile involvement. Clinicians with low suspicion order the GPP assay alone, and those who are unsure cast the broadest diagnostic net possible and order both assays, creating a data set to evaluate test performance characteristics and optimize the GPP assay. Tables 1 and 2 list the individual test results, demographic information, and inpatient versus outpatient status for data sets 1 to 3. A retrospective data review was performed using the UNC Health Care laboratory information system to identify patients with stool samples tested by the GPP and 2-SA from July 2013 to June 2014. These cotested samples (n = 1,105) (Table 2, data set 1) were analyzed by both assays utilizing either the same specimen or a second specimen collected within 24 h. The correlation of the 2-SA and its individual components (GDH EIA, toxin EIA, and reflex Xpert PCR) with GPP MFI values was independently analyzed. Receiver operator curves (ROC) were performed on data obtained from data set 1 utilizing GraphPad Prism to identify a GPP MFI value predictive of 2-SA and toxin EIA results. To examine the robustness and applicability of this MFI cutoff over time and between multiple reagent lots, equipment updates, and variations in testing personnel, we performed similar ROC analysis of data (Table 2, data set 2) obtained from a second group of patients (n = 1,432) with cotested stool samples analyzed between July 2014 and June 2015.
Patient characteristics by data set
Test results by data set
Determination of test performance characteristics.Table 3 highlights the approach taken to render a final interpretation for the presence of active infection for each cotested sample in data set 1. NAATs have high analytical sensitivity for the detection of an organism; therefore, samples negative for both the GPP (NAAT) and 2-SA (n = 917; GPP− and 2-SA−, respectively) were deemed negative for active infection. Detection of preformed toxin in stool correlates with clinical disease and patient outcome (27). Toxin EIAs target preformed toxin, and most exhibit very high specificity for active infection (12). The Alere QuikChek complete assay utilized in the current study exhibits >99% specificity, with cell cytotoxicity neutralization as the gold standard (12, 28). Although a subset of patients with toxin-positive EIA results may be asymptomatic (in vivo antibody mediated neutralization of toxin), detection of the etiologic agent of disease denotes metabolically active toxin-producing C. difficile, and patients in the toxin-positive EIA cohort (n = 56) were considered to have active infection without additional chart review. Patients with discordant test results including GPP−/PCR-positive (PCR+) (n = 7) and GPP-positive (GPP+)/2-SA− (n = 55) were adjudicated by chart review (described further below). GPP+/PCR+ cases (n = 70) were also adjudicated via chart review, given the ability of NAATs to detect colonization in the absence of active infection.
Interpretation of concordant results and adjudication of discordant results in cotested samples of data set 1a
Adjudication of discordant results via chart review.It is not possible to perform a thorough chart review without reading the results of C. difficile laboratory tests embedded within the patient notes and lab results contained in the medical record. Therefore, all clinical data were extracted from the medical record by two authors of the current study (S. M. Leal, Jr., and K. J. Levinson). The extracted clinical information was deidentified and removed of all information pertaining to C. difficile-specific test results for the encounter in question. The clinical vignette and associated lab parameters were independently analyzed by three infectious disease clinicians (B. Lehman, T. M. John, and M. B. Rios) blinded to the results of C. difficile laboratory tests utilizing the criteria outlined in Table 4 to render a clinical opinion (yes or no) on whether the patient was actively infected by C. difficile. The majority opinion (2/3) determined the final interpretation for that case. Defined criteria outlined in Table 5 were then utilized to categorize the severity of illness for each positive case. Disease severity stratification was used to identify actively infected patients in this cohort that would have benefited most from laboratory diagnosis. This study was not powered for definitive assessment of MFI correlation with disease severity. Likewise, although a subset of patients were more likely to be asymptomatically colonized by C. difficile (including immunocompromised individuals), this study was not powered to identify MFI thresholds uniquely predictive of active infection in these cohorts.
Screening criteria to identify active C. difficile infection by chart reviewa
Criteria to determine the severity of active C. difficile infection
Table 4 lists the set of criteria that must be met to categorize an active infection. The presence of ≥3 documented liquid stools per 24 h is required and was determined by reading clinician notes and nursing documentation in the medical record at the time the test was performed. In addition, the keyword search function was used in Epic to scan clinic notes and lab results within the appropriate time frame using keywords such as “bowel movements,” “BM,” “diarrhea,” and “stool.” Similarly, documented use of dysbiotic agents (antibiotics, chemotherapy, or immunosuppressants) within the past 2 months or a documented history of a prior C. difficile infection (≤6 months prior) or susceptible patient population (defined by inflammatory bowel disease [IBD], graft-versus-host disease [GVHD], cystic fibrosis [CF], or age ≥65 years) was determined by reading medical notes, medication history, problem lists, and searching with the keywords “antibiotics,” “infection,” “immunosuppressant,” “steroid,” “tacrolimus,” “biologics,” “recurrent,” and “difficile.” No laxative use within 48 h of symptom onset was determined by reading clinician notes and searching for documented usage up to 48 prior to the test date with the keywords “laxative,” “MiraLAX,” “Dulcolax,” “senna,” and “polyethylene glycol.”
The additional criteria listed in Table 4 are not required to categorize active infection but, rather, aid in the interpretation of specific clinical scenarios. Improvement on antibiotics with activity against C. difficile favors active infection, and this information was determined by reading initial clinician notes, searching for the prescription of relevant medications (flagyl, metronidazole, vancomycin, and fecal transplant), and reading follow-up notes to determine treatment efficacy. The absence of sick contacts with individuals with similar symptoms argues against readily transmissible GI pathogens and was determined by reading clinic notes and searching for “sick contacts.” The absence of another GI pathogen identified by laboratory assays also argues against alternative infections, and this information was obtained during the initial download of data sets 1 to 3. To rule out viral gastroenteritis, provider notes were read to identify cases in which emesis began prior to and exceeded diarrheal illness. Irritable bowel syndrome (IBS) was ruled out by searching for the terms “irritable bowel syndrome,” “IBS,” and “diarrhea” to detect any prior diagnosis of this condition and to identify patients with a long-standing history of recurrent mild diarrheal illness that resolved without C. difficile-specific treatment. Comparison of the current illness to prior diarrheal episodes was used to detect any change above baseline levels indicating active infection. Inflammatory bowel disease (IBD) flares were ruled out based on the frequency (above baseline level favors infection), consistency (more watery/malodorous than bloody favors infection), and symptom resolution upon antibiotic treatment (favors infection) and not immunosuppressant therapy alone (favors IBD).
Table 5 lists the criteria to categorize an active infection as either mild, moderate, severe, or fulminant. If diarrheal illness was the only active pathology (often the case for outpatients), then all systemic symptoms were attributed to it. If active comorbidities were present but symptoms initiated or worsened with the onset of diarrheal illness, they were attributed to the GI illness; otherwise criteria were not attributed to C. difficile infection. Data on the number of bowel movements (BMs) per 24 h were extracted from the medical record as described above. Mild cases exhibited ≥3 BM per 24 h, whereas moderate cases additionally exhibited systemic symptoms: temperature of >38°C or elevated creatinine (increased but <1.5× baseline) that initiated or worsened at the time of diarrheal illness. The presence and timing of systemic symptoms were determined based on clinician notes and searching Epic with the keywords “fever,” “vital signs,” and “creatinine” and reading through documents/lab results in the corresponding time frame. Severe cases additionally exhibited either radiologic, colonoscopic, or pathological evidence of pseudomembranous colitis or at least three of the listed criteria: elevated creatinine (>1.5× baseline), elevated lactate (increased but <5 mmol/liter), serum albumin (increased but <2.5 mg/dl), peripheral white blood cell (WBC) count of ≥15,000/mm3, an intensive care unit (ICU) stay attributable to the diarrheal illness, or age of ≥65 years. Patient age and peak lab results at the time of diarrheal illness were readily extracted from the medical record. The presence or absence of an ICU stay associated with the diarrheal illness and pseudomembranous colitis (PMC) was determined by reading provider notes and searching the key terms (“ICU,” “intensive care,” “pseudomembranous colitis,” “colitis,” and “PMC”). Fulminant infections were characterized by the presence of either toxic megacolon, death within 30 days due to diarrheal illness, or the presence of elevated serum lactate (≥5 mmol/liter) and a leukemoid reaction (>50/mm3). Provider notes and key term searches (“toxic megacolon” and “death”) were used to detect radiologic, colonoscopic, or pathological evidence of toxic megacolon and death.
Application of an MFI threshold-based reporting algorithm to detect C. difficile infection.We developed an algorithm (see Fig. 3) to report C. difficile results from the Luminex GPP based on high and low MFI threshold cutoffs and applied this reporting algorithm to a fresh data set (July 2013 to June 2014; n = 563) (Table 2) of patient samples analyzed by the GPP without 2-SA testing (indicative of low initial clinical suspicion). Next, we determined the percentage of samples that could be directly reported to treating clinicians as either positive or negative for the toxin gene based on the low MFI threshold (high sensitivity for organism), with a comment indicating organism burden suggestive of (or indeterminate for) active infection based on the high MFI threshold (high specificity for active infection). Chart reviews were performed on a subset of patients in data set 3 to determine the number of active infections in patients with MFI values of ≥1,200, and 15/21 medical records were available for review.
Statistics.GraphPad Prism (San Diego, CA) was utilized to assess the statistical significance of GPP MFI values corresponding to specific experimental variables. Nonparametric two-tailed t tests (Mann-Whitney test) were used to analyze the statistical significance between two data groups. Nonparametric one-way analysis of variance (ANOVA; Kruskal-Wallis test) with Dunn's posttest analysis was used to analyze the significance between >2 experimental variables. Test performance characteristics (concordance, sensitivity, specificity, positive predictive value [PPV], and negative predictive value [NPV]) were calculated via standard methods. The Clopper-Pearson method was used to calculate 95% confidence intervals for each test performance characteristic. The statistical significance of test performance characteristics was determined using McNemar's test, and all P values of ≤0.05 were considered statistically significant.
Study approval.This study was approved by the Institutional Review Board at the University of North Carolina at Chapel Hill (Chapel Hill, NC).
RESULTS
Correlation of GPP MFI thresholds with 2-SA and toxin EIA positivity.To test the hypothesis that GPP MFI thresholds could be utilized to predict 2-SA and toxin EIA positivity, we analyzed 1,105 cotested samples (data set 1) with the individual test results, demographic information, and inpatient versus outpatient status shown in Tables 1 and 2. In this study, we sought to identify an MFI cutoff with high specificity and therefore averaged toxin A and B (A/B) MFI values to reduce false positives (toxin A, 1/1,105, or 0.1%; toxin B, 3/1,105, or 0.3%) caused by high MFI values associated with a single toxin.
Figure 1A shows a dot plot of averaged toxin A/B MFI values for 2-SA-negative and 2-SA-positive cases. Individual data points, the mean, and 95% confidence intervals (indicated by the error bars) are highlighted for each experimental variable. 2-SA-positive cases exhibited statistically significant increases (P < 0.05) in MFI values compared with those of 2-SA-negative cases. Visual inspection of this graph shows an MFI cutoff of ≥1,200 (indicated by the dotted line) that accurately classified 962/973 (99%) 2-SA-negative cases and 105/132 (80%) 2-SA-positive cases. Figure 1B shows a receiver operator curve (ROC) of GPP MFI values as a function of 2-SA results with a statistically significant (P < 0.05) area under the curve (AUC). Data points representing the standard GPP MFI cutoff value of 150 and the more specific MFI cutoff value of 1,200 are also indicated. Figure 1C shows a dot plot of averaged toxin A/B MFI values for individual components of the 2-SA (GDH antigen negative, reflex PCR negative[PCR−], toxin-positive EIA, and reflex PCR+). MFI values associated with toxin EIA+ and PCR+ cases show statistically significant differences (P < 0.05) compared to values for GDH− and PCR− cases. Application of an MFI cutoff of ≥1,200 (indicated by the dotted line) accurately identified 897/908 (99%) of GDH− cases, 64/64 (100%) of PCR− cases, 53/56 (95%) of toxin-positive EIA cases, and 52/76 (68%) of PCR+ cases. Figure 1D shows an ROC of MFI values as a function of toxin EIA results with a statistically significant AUC (P < 0.05) and highlighted data points representing the standard MFI cutoff of ≥150 and the high-specificity MFI cutoff of ≥1,200. Although multiple cutoffs were evaluated, the high specificity and acceptable sensitivity of an MFI cutoff of ≥1,200 were deemed optimal to predict 2-SA and toxin EIA positivity.
Analysis of cotested samples identifies a GPP quantitative MFI threshold of ≥1,200 as predictive of 2-step algorithm and toxin EIA results. (A) GPP averaged toxin A/B MFI values corresponding to 2-SA-negative and -positive cases. (B) Receiver-operator curve (ROC) analysis of GPP MFI values as a function of 2-SA results. (C) GPP MFI values corresponding to the results of individual components in the 2-SA, including GDH antigen, reflex PCR, and toxin EIAs. (D) ROC analysis of GPP MFI values as a function of toxin EIA results. Individual data points, the mean, and 95% confidence intervals indicated by the error bars are shown for each experimental variable. Std, standard.
Correlation of the MFI cutoff of ≥1,200 to 2-SA and toxin positivity in a second cotested data set.To examine the robustness and applicability of MFI thresholds over time and between multiple reagent lots, equipment updates, and variations in testing personnel, we applied this cutoff to a second group of patients (data set 2; n = 1,428) with cotested stool samples. Figure 2A shows a dot plot of averaged toxin A/B MFI values for 2-SA-negative and 2-SA-positive cases. Differences in MFI values between both groups are statistically significant (P < 0.05), and an MFI cutoff of ≥1,200 accurately classified 1,258/1,278 (98%) of 2-SA-negative cases and 121/150 (81%) of 2-SA-positive cases. Figure 2B shows an ROC of MFI values as a function of 2-SA results with a statistically significant AUC (P < 0.05) and similar localization of MFI values. Figure 2C shows a dot plot of averaged toxin A/B MFI values for individual components of the 2-SA with statistically significant differences (P < 0.05) in MFI values associated with toxin-positive and PCR+ cases compared to GDH− and PCR− cases. Application of an MFI cutoff of ≥1,200 accurately classified 1,193/1,213 (98%) GDH− cases, 65/65 (100%) PCR− cases, 56/63 (89%) toxin-positive cases, and 65/87 (75%) PCR+ cases. Figure 2D shows an ROC of MFI values as a function of toxin EIA results with a statistically significant AUC (P < 0.05) and similar localization of MFI values. These data indicate that the ability of an MFI cutoff of ≥1,200 to predict 2-SA and toxin EIA positivity is robust over time and minimally susceptible to normal testing variations (multiple reagent lots, equipment updates, variations in testing personnel, etc.) in the clinical laboratory.
Application of the GPP MFI cutoff of ≥1,200 to a second cotested data set is predictive of 2-step algorithm and toxin EIA results. (A) GPP averaged toxin A/B MFI values corresponding to 2-SA-negative and -positive cases. (B) ROC analysis of GPP MFI values as a function of 2-SA results. (C) GPP MFI values corresponding to the results of individual components in the 2-SA. (D) ROC analysis of GPP MFI values as a function of toxin EIA results. Individual data points, the mean, and 95% confidence intervals indicated by the error bars are shown for each experimental variable.
Adjudication of discordant samples via chart review.Table 3 outlines the approach taken to render final interpretations for all cotested samples in data set 1 (n = 1,105) and the results of chart review adjudication. GPP data used to render final interpretations were obtained utilizing the standard MFI cutoff of ≥150 to achieve optimal sensitivity and rule out the presence of a C. difficile strain encoding toxin genes in the patient's stool sample (no organism equates to no active infection). Chart reviews were performed to adjudicate discordant test results and GPP+/reflex PCR+ cotested samples. During the chart review process, 6/132 individuals were excluded due to age of <3 years (3 GPP+/GDH− and 3 GPP+/PCR+), and 7/132 patients (3 GPP+/GDH− and 4 GPP+/PCR+) were excluded due to inaccessible medical records. The extracted clinical data from completed chart reviews (n = 119) were interpreted by infectious disease (ID) clinicians blinded to the results of laboratory tests, and the majority opinion (2/3) determined the final interpretation for that case. Unanimous agreement between all three clinicians was observed for 90/119 (76%) of cases. There were no specific trends observed for the subset of patients (n = 29) for which a unanimous agreement was not reached. Of the 7 GPP−/PCR+ cases, 4 were deemed negative and 3 were mild positives by chart review. The following adjudication results were noted for the 54 GPP+/GDH− cases: 38 negative, 11 positive (3 mild, 7 moderate, and 1 fulminant), and 6 excluded (due to age or inaccessible medical record). The single GPP+/PCR− sample was deemed a mild positive, and the 70 GPP+/PCR+ cases exhibited the following adjudication results: 20 negative, 42 positive (27 mild, 13 moderate, and 2 severe), and 7 excluded. These data (n = 1,092) were used to determine the test performance characteristics (concordance, sensitivity, specificity, negative predictive value, and positive predictive value) of each assay.
Test performance characteristics of the 2-SA and GPP MFI thresholds.Table 6 gives the test performance characteristics of the GPP interpreted with an MFI cutoff of ≥150 (standard), an MFI cutoff of ≥1,200, and the 2-SA (respectively): concordance, 94.5, 95.9, and 96.7%; sensitivity, 93.0, 77.9, and 89.4%; specificity, 94.7, 98.0, and 97.6%; positive predictive value, 66.9, 81.5, and 80.8%; and negative predictive value, 99.2, 97.5, and 98.8%. McNemar's test identified statistically significant differences (P < 0.05) in concordance between all three laboratory assays, decreased sensitivity of the MFI cutoff of ≥1,200, and decreased specificity of the MFI cutoff of ≥150 compared with results of alternative assays. Table 7 shows that increasing the MFI threshold cutoff from the standard of ≥150 to ≥1,200 eliminated 32 false positives at the expense of an additional 17 false negatives and that an MFI cutoff of ≥150 produced 28 more false positives and 4 fewer false negatives than the 2-SA. Table 7 also shows that raising the MFI cutoff to ≥1,200 results in 4 fewer false positives at the expense of 13 additional false negatives compared to results with the 2-SA.
Sensitivity and specificity of the GPP versus the 2-step algorithma
Comparison of GPP and two-step algorithm results and clinical results
Development and application of a GPP MFI threshold-based reporting algorithm.Given the high sensitivity of the MFI cutoff of ≥150 and high specificity of the MFI cutoff of ≥1,200, both thresholds were incorporated into a reporting algorithm (Fig. 3) to report C. difficile results from the GPP assay with similar performance characteristics as the 2-SA. In this algorithm, samples with MFI values of <150 are reported as negative for detection of the C. difficile toxin gene, with 93% sensitivity and 99% NPV. Samples with MFI values of ≥1,200 and no other pathogen detected are reported as positive for detection of the C. difficile toxin gene and as having an organism burden suggestive of active infection, with 98% specificity and 82% PPV. Samples with MFI values in the intermediate range (>150 but <1,200) or MFI values of ≥1,200 with concomitant detection of a second pathogen (possible false positive) are reported out as positive for detection of the C. difficile toxin gene with additional clinical scenario-dependent reflex testing. Samples destined for reflex testing are known to harbor the organism (NAAT positive), and further adjudication with a high-specificity assay (e.g., toxin EIA) is indicated to detect active infection. Positive or negative adjudicated results should be reported as either suggestive of or indeterminate for active infection, respectively.
Proposed algorithm to interpret GPP data enabling organism detection with high sensitivity (93% if MFI ≥150) and active infection with high specificity (98% if MFI ≥1,200). An MFI cutoff of ≥1,200 yields 96% concordance with the 2-step algorithm and 98% specificity for active infection (∧). An MFI cutoff of <150 yields a 99% negative predictive value for active infection (#). Samples destined for reflex testing (∼) are positive for the organism (as detected by NAAT) and should be analyzed by an assay with high specificity for clinical disease (e.g., toxin EIA).
In Fig. 4, we present data on reporting outcomes when this algorithm is applied to 563 samples (Table 2) without concomitant 2-SA testing (indicative of initial low clinical suspicion). We show that 535/565 (95%) samples would have been immediately reported to treating clinicians as either negative (514; 91%) or positive for C. difficile at an organism burden suggestive of active infection (21; 4%) without the need for additional testing. Of the 514 samples testing negative for C. difficile, 86 (17%) tested positive for a GI pathogen. Chart reviews performed on patients with MFI values of ≥1,200 identified 5 patients with active infection (3 mild, 1 moderate, and 1 severe) that did not receive C. difficile-specific testing. Samples with MFI values in the intermediate range (23; 4%) or MFI values of ≥1,200 with concomitant detection of a second pathogen (5; 0.9%) would have been reported out as positive for C. difficile, alerting clinicians to the possibility of active infection and availability of reflex toxin EIA testing if warranted by the clinical scenario.
Algorithmic interpretation of GPP data enables accurate detection of infections from samples sent only for syndromic panel testing, irrespective of clinical suspicion for C. difficile. Samples with an MFI of ≥1,200 (∼4%) or <150 (∼91%) are reported to treating clinicians as positive or negative, respectively, without the need for additional testing (#). Samples with MFI values in the intermediate range (≥150 but <1,200; ∼4%) and samples positive for both C. difficile and a second pathogen (∼0.9%) are reported as positive for the C. difficile toxin gene with optional reflex toxin EIA testing (∧). Of the 514 samples testing negative for C. difficile, 86 (17%) tested positive for other pathogens (∼). Chart reviews identified 5/16 patients with undiagnosed active infection (3 mild, 1 moderate, and 1 severe) (*).
DISCUSSION
Classic presentations of C. difficile infection elicit high clinical suspicion and prompt providers to order laboratory tests specifically targeting this pathogen (1). However, presentations with symptom onset in the community and incomplete knowledge of associated risk factors result in up to 60% of infected outpatients not receiving tests capable of accurately detecting C. difficile (16, 18). This study aimed to fill this diagnostic void by identifying and applying quantitative cutoffs to the interpretation of data from the Luminex GI pathogen panel, enabling accurate detection of C. difficile infections from stool samples irrespective of clinical suspicion.
The need for reporting C. difficile from molecular syndromic panels is evident from recent reports on the epidemiology of CA-CDI. Hensgens et al. analyzed stool samples (n = 12,714) submitted from outpatient settings in the Netherlands with either toxin EIAs or cell cytotoxicity neutralization assays (CCNA) and identified a CA-CDI rate of 1.5% (n = 191; the same rate as Salmonella gastroenteritis), with 60% (n = 115) of these cases lacking clinician orders for C. difficile testing (18). Even if a careful history had captured all patients with recent antibiotic exposure or hospitalization, 39% (n = 75) would have remained untested (18). Likewise, Wilcox et al. identified a CA-CDI rate of 2.1% (by CCNA) in the United Kingdom, with 33% of patients lacking exposure to antibiotics (15). In the United States, stool samples submitted from emergency departments and outpatient clinics between 2002 and 2007 were prospectively analyzed by toxin EIAs, yielding a CA-CDI rate of 3.9%, with 37% lacking recent antibiotic exposure and 67% lacking hospitalization within the previous month (14). Since the introduction of stand-alone NAATs with increased sensitivity, epidemiologic data have indicated an increased rate of CA-CDI (17). Utilizing data obtained in 2011, the CDC estimated an annual incidence rate of 453,000 total cases of C. difficile in the United States, of which 15.8% (n = 71,574) exhibited symptom onset in the community and lacked documented inpatient health care exposure (16, 17). Given the high sensitivity of NAATs, the true incidence of CA-CDI is likely on the lower end of the spectrum of published figures, i.e., between 1.5 and 15.8% (15–17, 29). Although our study was not designed for epidemiologic purposes, we show a positivity rate consistent with the literature (4% to 9%) in stool samples submitted for GPP analysis without directed testing for C. difficile.
The ability to report C. difficile from a syndromic panel is meant to supplement, not replace, directed-testing strategies. If clinical suspicion is high, targeted assays (2-SA and NAATs) should continue to be utilized as they are cheaper and faster, with comparable or better test performance characteristics and improved reimbursement rates (1, 12, 30, 31). If a clinician orders both directed testing and multiplex PCR analysis, C. difficile data could be reported out from both assays or reported solely from the directed-testing strategy. This decision would need to be made by individual lab directors as discordant results can lead to medico-legal/ethical dilemmas concerning whether to report a known positive test result. However, in the not infrequent situation in which low clinical suspicion triggers an order for multiplex PCR analysis but not directed testing, we propose that C. difficile data be reported from the Luminex GPP assay via the algorithm outlined in Fig. 3.
This algorithm takes advantage of a low MFI cutoff of ≥150 to accurately report detection of the organism with high analytic sensitivity (93%) and uses a high MFI cutoff of ≥1,200 to detect bacterial burden suggestive of active infection with high specificity (98%). The application of quantitative thresholds to interpret molecular data in the current study is similar to recent work utilizing PCR cycle threshold (CT) values to predict organism burden (21, 23), toxin EIA positivity (24), and patient outcome (20, 22). However, the current study is the first to apply quantitative cutoffs to a molecular syndromic panel enabling the accurate identification of C. difficile infection to the same extent as directed-testing strategies and the detection of patients with CA-CDI that would otherwise go undetected. Although most of these cases would resolve on their own (as with most diarrheal illness), a subset of patients with moderate to severe disease may have likely benefited from antimicrobial therapy with faster symptom resolution, decreased morbidity, prevention of serious sequelae, reduced risk of spreading the disease, and increased clinical suspicion for C. difficile should symptom onset recur (19). In the inpatient setting, although the GPP assay has a longer turnaround time than directed-testing strategies, the ability to accurately identify patients with active infection who would otherwise not receive testing for C. difficile enables initiation of appropriate infection prevention measures, mitigating further propagation of the organism throughout the health care facility. Although our findings are specific to the Luminex GPP, it is possible that a similar approach could be utilized to report C. difficile from other multiplex PCR assays (13).
The algorithmic approach to reporting C. difficile from the GPP assay can be tailored to the philosophy of the individual institution. NAAT-only advocates utilizing strict preanalytic restrictions recommended by guidelines of the Infectious Diseases Society of America (IDSA) (32) may choose to take advantage of the high sensitivity of the GPP and report only detection of the C. difficile toxin gene for samples with MFI values of ≥150. In this study, we observed 52 false positives using an MFI cutoff of ≥150 in cotested samples in contrast to 20 and 24 for an MFI cutoff of ≥1,200 and the 2-SA, respectively. If this approach were to be utilized, strict ordering restrictions (difficult to enforce) that limit testing to patients with unexplained new-onset unformed stools (3/24 h) who are not on laxatives are highly recommended to limit false positives, given the relatively lower pretest probability of active infection in this cohort. Institutions seeking to mitigate false positives can choose to report samples with MFI values of ≥1,200 as positive with specificity and PPV values similar to those of the 2-SA (specificity, 98.0 versus 97.6%, respectively; PPV, 81.5 versus 80.8%, respectively).
Institutions that seek the benefits of high sensitivity for organism detection and high specificity for active infection can utilize the proposed reporting algorithm shown in Fig. 3. Samples with MFI values of <150 are reported as negative for the organism with a high NPV (99%), and those with MFI values of ≥1,200 are reported as positive for the organism, with a comment on bacterial burden suggestive of active infection (specificity, 98%). Samples in the intermediate category with an MFI of ≥150 but <1,200 should be reported as positive for organism detection, with treating clinicians alerted to the option for reflex toxin EIA (high specificity for active infection) testing if warranted by the clinical scenario (1, 12). The main benefit of this algorithmic approach is that treating clinicians receive the benefit of knowing if the organism is present utilizing the most sensitive testing strategy (NAAT; MFI of ≥150) and receive either an immediate estimate of organism burden indicating that the patient is likely actively infected (MFI of ≥1,200) or the option for reflex testing to detect the etiologic agent of disease (toxin protein; MFI of ≥150 but <1,200). Clinicians can then correlate these data with the clinical scenario to make more informed decisions affecting patient care.
Establishing quantitative cutoffs to interpret molecular data is pointless if the numerical values corresponding to the outcome of interest fluctuate over time. In this study, we show that the MFI cutoff of ≥1,200 maintained its ability to accurately predict 2-SA and toxin EIA positivity in a second cotested data set (n = 1,432) with samples analyzed 1 year apart using multiple reagent lots, equipment updates, and variations in testing personnel. The robustness of MFI thresholds in the current study is similar to the high precision observed for Xpert PCR CT values (24) although additional studies at multiple testing sites are required for a more thorough evaluation.
Analysis of data set 1 reveals that out of 1,670 samples tested with the GPP assay over a 1-year period, 1,440 (86%) would have been reported out as negative utilizing the proposed algorithm. Had the option to report C. difficile results from the GPP been available at the time, no doubt a significant subset of the 1,105 (66%) cotested samples would have been analyzed by the syndromic panel alone without the need for additional testing. If GPP ordering had been combined with proper lab test utilization control methods (33), such as electronic hard stops and house staff education, the ability to report C. difficile from the GPP would have ultimately reduced costs associated with cotesting (13, 34).
Implementation of C. difficile reporting from syndromic panels as an adjunct to directed-testing strategies also has the potential to raise health care costs. If given the option of obtaining C. difficile results from either assay, many clinicians, especially those with low to intermediate concern for C. difficile infection, would likely choose the more expensive syndromic panel to simultaneously detect other potential pathogens. Laboratories that choose to report C. difficile results from both assays should be aware of the potential for abuse and take the appropriate steps to mitigate inappropriate lab test utilization. Provider education about the cost of laboratory tests and electronic hard stops during the test ordering process should emphasize (i) that directed-testing strategies are the more accurate and cost-effective method to detect C. difficile, (ii) that testing should not be performed on patients exposed to laxatives due to high false-positive rates, and (iii) that director approval should be required for inpatients hospitalized for >72 h (33).
It is important that in addition to the 21 positive cases of C. difficile, 86 other pathogens were identified and reported out from the 563 samples analyzed by the GPP alone. These pathogens are listed in order of increasing frequency: norovirus, Campylobacter, Salmonella, Escherichia coli STEC/O157, Giardia, Shigella, rotavirus, and Cryptosporidium. The overlapping clinical syndromes associated with these organisms makes multiplex PCR a powerful technology in the diagnosis of GI infections, and its utilization in clinical laboratories continues to increase despite high costs and reimbursement challenges (13). Similar to published reports, we also observed a significant number of samples with codetection of C. difficile and a second pathogen (25, 26, 35). It is not clear if codetection represents true coinfection or detection of asymptomatic carriage in a patient with diarrheal illness due to the other pathogen (4). However, chart reviews performed on six samples with codetection of C. difficile and a second pathogen (four norovirus, one Campylobacter, and one adenovirus by PCR) identified all as false positives, suggesting bystander detection and not active infection.
In summary, although directed-testing strategies are the preferred diagnostic approach, their value is contingent upon clinical suspicion for C. difficile infection. Medicine is as much an art as it is a science, and as recently shown, up to 60% of community onset cases will not elicit an order for C. difficile testing (18). The attraction of this algorithmic approach to reporting C. difficile results from multiplex PCR assays is that it serves as an adjunct to supplement, not replace, directed-testing strategies while simultaneously capturing atypical infections, irrespective of initial clinical suspicion, and a plethora of GI pathogens that cause the same nondescript diarrheal syndrome.
ACKNOWLEDGMENT
Luminex technologies provided funding but did not have a role in study design, data collection and interpretation, or the decision to submit the work for publication.
FOOTNOTES
- Received 29 November 2017.
- Returned for modification 27 December 2017.
- Accepted 26 March 2018.
- Accepted manuscript posted online 11 April 2018.
- Copyright © 2018 American Society for Microbiology.
