INTRODUCTION

Top Abstract Introduction Materials & Methods Results Discussion References

Aspergillus species are saprophytic filamentous fungi which may cause invasive pulmonary infections in patients with compromised host defenses. The number of patients infected with this organism is increasing, and in some institutes Aspergillus species have now become the most frequent cause of invasive fungal infection (5). Establishing a diagnosis is often difficult, and the majority of infections remain undetected during life (5). The performance of several new diagnostic tests and procedures is under investigation, including the detection of Aspergillus DNA by PCR (2, 8, 9, 19) and of fungal antigens by serological methods (11, 17). A commercially available sandwich enzyme-linked immunosorbent assay (ELISA; Platelia Aspergillus; Sanofi Diagnostics Pasteur, Marnes-La-Coquette, France) which detects galactomannan (14), a cell wall constituent of Aspergillus species which is released during growth (13), has been developed. The assay has been evaluated in several institutes with serum mainly from patients with hematological malignancies, and a high sensitivity of 67 to 100% and a specificity of 81 to 100% were found, especially when a series of serum samples was used (10, 14, 15, 21). The detection of galactomannan at an early stage of disease (10, 15, 21, 22) and the quantitative titers produced by the sandwich ELISA offer new approaches to the management of invasive aspergillosis (12). Sera from patients may be screened for the presence of galactomannan during periods of high risk of invasive aspergillosis (12), and once galactomannan is detected, the antigen titer may be monitored during antifungal treatment since preliminary investigations have shown that at least in some patients the course of the titer corresponded to the clinical outcome (10, 23). As a consequence, series of serum samples from single patients collected at regular intervals over a long period of time will be tested by sandwich ELISA. However, although data on the accuracy of the assay are accumulating, little is known about the reproducibility of the sandwich ELISA, both within one laboratory and between laboratories. The present investigation was performed to establish the intra- and interlaboratory reproducibilities of the sandwich ELISA in order to develop tentative quality control guidelines for using the sandwich ELISA as a tool for monitoring serum galactomannan levels in patients at high risk for invasive aspergillosis.

	MATERIALS AND METHODS

Top Abstract Introduction Materials & Methods Results Discussion References

Sample preparation. Serum samples from 12 patients who had been admitted to the Hematology Department of the University Hospital Nijmegen were used to prepare 20 test samples. The characteristics of the patients and the samples used are presented in Table 1. The serum samples had been routinely tested by sandwich ELISA and stored at 80°C. Since 4.2 ml of each test sample was required, serum from single patients samples were pooled. For 19 test samples no more than two consecutive samples were pooled, and for one test sample three samples were pooled. The 20 test samples included eight samples with ELISA reactivity and eight ELISA-negative samples (Table 1). For three patients both ELISA-negative and ELISA-positive serum samples were available (Table 1, patients 1, 2, and 3). Among the test samples were included four ELISA-positive samples which were thought to be false positive because a single positive sample was obtained among a series of at least six negative samples (Table 1, patients 9, 10, and 11) or consecutive samples were positive for a patient with no clinical or radiological evidence of invasive aspergillosis (Table 1, patient 12). In addition to these clinical samples eight serum samples without ELISA reactivity were prepared; these samples were spiked with known concentrations of galactomannan, i.e., 0, 0.35, 0.7, 1.0, 1.5, 2.0, 2.5, and 5.0 ng of galactomannan per ml. For each of the 28 test samples 700-µl aliquots were prepared and frozen at 80°C and were sent to the participating laboratories on dry ice.

Sandwich ELISA. The sandwich ELISA was performed as described previously (14). Briefly, 300 µl of each sample was mixed with 100 µl of treatment solution and the mixture was subsequently boiled for 3 min. After centrifugation, the supernatant was used for further testing. Fifty microliters of conjugate was added to each well of an anti-galactomannan immunoglobulin M-coated microtiter plate (Platelia Aspergillus; Sanofi Diagnostics Pasteur), followed by the addition of 50 µl of the treated sample. After 90 min of incubation at 37°C, the plates were washed and 100 µl of substrate buffer containing ortho-phenylenediamine hydrochloride was added to each well, and the plates were incubated for 30 min at room temperature in darkness. The optical density at 450 nm was measured, and each plate contained a positive control (5 ng of galactomannan per ml), a threshold control (1 ng of galactomannan per ml), and a negative control (no galactomannan). The optical density of the threshold control was recommended by the manufacturer to be between 0.3 and 0.8, the ratio between the negative control and the threshold control was recommended to be below 0.5, and that of the positive control and the threshold control was recommended to be greater than 2. The ratio between the optical density of the test sample and that of the threshold control was calculated for each serum sample. A ratio of less than 1.0 was negative, a value greater than 1.5 was positive, and those between 1.0 and 1.5 were undetermined (gray zone), which is recommended by the manufacturer.

Study design. Six laboratories (coded as laboratories 1, 2, 3, 4, 5, and 6) of university hospitals in The Netherlands participated in the study. Each laboratory received the same 28 blinded test samples in 700-µl aliquots and a detailed protocol. Twenty-four sandwich ELISA plates from a single batch were kindly donated by the manufacturer, and four plates were sent to each participating center. Each laboratory used two plates to familiarize the investigators with the assay, and the remaining two plates were used to test 20 test samples in duplicate on 2 different days. The eight serum samples which were spiked with galactomannan were tested in duplicate in a single run at five institutes. The assays were performed within 4 weeks of receipt of the samples and plates. Each institute reported the optical density, the ratio calculated for each serum sample, and the interpretation of the result. Thus, a total of 24 optical density values were available for each of the 20 test samples and 10 were available for each of those spiked with galactomannan.

Statistical analysis. The objectives of this study were (i) to determine the intra- and interassay variabilities of the sandwich ELISA with identical serum samples and microtiter plates in each of six laboratories, (ii) to determine the interlaboratory reproducibility of the sandwich ELISA with respect to both quantitative and qualitative assay results, (iii) to calculate the optimal cutoff values for negative and positive samples and compare the calculated cutoff values to those recommended by the manufacturer, and (iv) to determine the level of agreement between the known and measured concentrations of galactomannan in serum.

	RESULTS

Top Abstract Introduction Materials & Methods Results Discussion References

For the clinical samples a total of 480 optical density values were evaluated, e.g., 192 for each of the ELISA-positive and -negative samples and 96 for the false-positive samples. Two of the false-positive samples showed ELISA reactivity, while the remaining false-positives samples were negative; thus, 10 samples showed ELISA reactivity and 10 samples were negative. The ranges of the optical density values and the ratios for the serum samples tested were 0.028 to 0.474 and 0.15 to 1.21, respectively, for the negative samples and 0.446 to 4.849 and 1.40 to 12.69, respectively, for the positive samples. Among the positive samples, because of one sample with the highest concentration of galactomannan, 9 of 24 optical density measurements were out of range. These off-scale optical density results were excluded from further analysis, leaving a total of 471 optical density values. The eight serum samples spiked with galactomannan were analyzed at five institutes, resulting in 80 optical density values.

Control samples. The optical density value of the threshold control is critical to the calculated ratio for the serum sample result since the optical density value is used as a denominator. The reproducibilities of the optical densities of the threshold control samples obtained in duplicate are presented in Fig. 1. Although the duplicate values of the optical density were reproducible, 9 of 24 (38%) measurements were not within the recommended limits. Duplicate measurements of the optical density values of the threshold control were within the recommended limits on both days in only one laboratory. In one institute the optical density of the threshold control was above the upper limit, which resulted in low ratios for the serum samples. For this threshold control sample the optical density was measured 2 h after the optical densities of the test samples were measured. Therefore, only the optical density values of these serum samples and not the calculated ratios for serum were included in the analysis. The remaining 22 optical density values were below 0.5, including 7 that were below the lower limit of 0.3 (Fig. 1). Nevertheless, the ratios for serum, which were calculated by using these low threshold control values, were not significantly different from those calculated by using threshold control values within the recommended limits. The mean optical density of the 24 threshold control measurements was 0.38 (range, 0.24 to 0.82), which was significantly lower than the mean optical density of the threshold controls for 20 consecutive microtiter plates routinely used in one institute (mean optical density, 0.57; range, 0.24 to 0.83; P < 0.001).

View larger version (9K): [in this window] [in a new window]   FIG. 1.   Reproducibility of duplicate optical density measurements by sandwich ELISA of the threshold control which contains 1 ng of galactomannan per ml. For each institute optical density values measured on two different days are shown. , institute 1; , institute 2; , institute 3; , institute 4; ×, institute 5; +, institute 6. The window represents the recommended limits of the optical density of the threshold control. The points situated inside the window represent threshold control measurements of which both (duplicate) optical density values were within the recommended limits.

Calculation of cutoff values. Ninety-five percent of the optical density measurements of 10 serum samples without ELISA reactivity were below 0.76, 99% were below 0.97, and 99.8% were below 1.14. These results indicate that the cutoff values for negative, gray zone, and positive test results should be <0.8, 0.8 to 1.0, and >1.0, respectively.

Reproducibility. The interassay reproducibility of the threshold control was calculated by using the CV. The mean CV was 9.5% for repeat testing of a single threshold control measurement and 6.7% for repeat testing of duplicate measurements. The intra- and interlaboratory reproducibilities for the test samples are presented in Table 2.

Serum samples spiked with galactomannan. The correlation coefficients between the known and measured galactomannan concentrations were 0.975, 0.957, 0.987, 0.796, and 0.883 for institutes 1, 2, 3, 4, and 6, respectively. Overall, the mean correlation coefficient was 0.949 (95% confidence interval, 0.918 to 0.968).

	DISCUSSION

Top Abstract Introduction Materials & Methods Results Discussion References

Until recently, the detection of Aspergillus antigens has not played a major role in the diagnosis of invasive aspergillosis in neutropenic patients. This is partly due to the low sensitivity of commercial tests such as the latex agglutination test (Pastorex Aspergillus; Sanofi Diagnostics Pasteur), which detects the Aspergillus antigen galactomannan (6, 20, 21). Since in most cases antigen was detected at an advanced stage of infection, the latex agglutination test did not contribute to the decision to treat patients with antifungal agents (6, 20). The development of a sandwich ELISA with a significantly improved sensitivity offers new approaches to managing the disease (12). Several studies performed in Europe have shown that the sandwich ELISA contributes to the early diagnosis of invasive aspergillosis and adds to the certainty of the diagnosis, at least for patients receiving cytotoxic treatment for hematological malignancies (10, 15, 22). Since the optimal use of the sandwich ELISA probably involves analysis of a series of serum samples from individual patients, we investigated the reproducibilities of the assay both within and between runs in a single institute and between institutes.

Our investigation indicates that the cutoff values which are recommended by the manufacturer may be too high. On the basis of our results, cutoff values for negative, gray zone, and positive ELISA results should be <0.80, 0.80 to 1.0, and >1.0, respectively. Samples with values between 0.80 and 1.0 would require retesting. Previous studies have also suggested that lower cutoff values can be used. In one study the threshold of positivity was estimated to be 0.93 ng of galactomannan per ml on the basis of more than 100 measurements for children without invasive aspergillosis (10). In another study the threshold of positivity was estimated to be 0.8 from the mean optical density plus 5 SDs, which corresponds to a concentration of 1 ng of galactomannan per ml (15). However, for both studies it remains unclear if the controls were healthy blood donors or neutropenic patients without invasive aspergillosis. We found that the mean optical density among blood donors was significantly lower than that among neutropenic patients without evidence of invasive aspergillosis (3), and therefore, the choice of cutoff value may depend on the type of high-risk population under investigation. Both the number of false-positive ELISA results and the benefit of earlier positivity should be taken into account in choosing a cutoff value. For instance, lowering of the cutoff value from 1.5 to 1.0 may not result in an earlier positivity of the ELISA if the patients are sampled and analyzed once weekly, and therefore, the delay between diagnosis and treatment of the infection will not be shortened significantly. The manufacturer of the ELISA proposes that ratios for serum of between 1.0 and 1.5 be interpreted as nonconclusive (gray zone) regarding the absence or presence of infection. Further studies are needed to study optimal cutoff values in more detail by using receiver operating characteristic curves. For clinical purposes the result for a serum sample with a ratio within the gray zone is sometimes difficult to interpret. The ELISA result could be weakly positive due to inadequate storage of the serum sample (20), cross-reactivity with an unknown component in the serum sample (15, 16), or laboratory contamination (7), especially with airborne dust, which may carry spores from Aspergillus species or Penicillium species. Alternatively, the weakly positive ELISA result may reflect a gradual rise in the galactomannan titer in the patient. In both cases the sample should be retested together with a second sample from the patient, which is also recommended by the manufacturer (16). If both samples or only the second sample is ELISA negative, an infection in the patient is unlikely, while a rising titer in the serum strongly suggests the presence of an infection. Nevertheless, for some patients weakly positive ELISA results may be found for consecutive serum samples, and since at present the reason for this remains unclear, careful clinical and radiological observation of the patient, together with further monitoring of the serum, is required. In general, the interpretation of ELISA results is facilitated if series of serum samples from individual patients are available.

The reproducibility of the optical density of the threshold control among the six laboratories was good if the sample was tested in duplicate, which is also recommended by the manufacturer. The reproducibility of the sandwich ELISA was excellent when the results for the clinical samples were analyzed as ordinal data. The high K_w indicates that during repeated testing the samples were classified into the same diagnostic category. When it comes to clinical decision making, in most cases a categorical interpretation of the ELISA result is sufficient, since a test that detects antigen in the serum is usually only one of several diagnostic tests or procedures which are used to make the diagnosis. The presence of highly indicative signs by high-resolution computed tomography of the chest of a neutropenic patient combined with the ELISA reactivities of two serum samples strongly supports the diagnosis of invasive aspergillosis, irrespective of how high the actual ratios for the serum samples are. However, since false-positive results may occur, in some cases the course of the antigen titer may help to distinguish between false-positive reactivity and true antigenemia (16). Furthermore, for a number of patients the course of the antigen titer during antifungal therapy corresponds to treatment outcome (10, 23). To this end the ELISA results will be interpreted as continuous data and antigen levels from consecutive measurements will be compared. Although the diagnostic accuracy of the ELISA, as calculated by the use of kappa statistics, is very good, the analytical repeatability is not optimal. The correlation coefficients found for serum samples spiked with galactomannan indicate a suboptimal analytic recovery of the added galactomannan. This is not unexpected in the face of the considerable assay variability. The optical density values of the clinical samples showed considerable variation between runs within a single institute, even though in this study all samples were analyzed with ELISA plates from a single batch. For example, the ratio for a serum sample of 6 measured on one day could vary between 3.78 and 8.22 (95% confidence interval) when measured on another day. Of course, many factors may contribute to the observed variation, including pipetting errors, efficacy of washing of the microtiter plates, and the experience of the technician. Alternatively, the conditions of coating of the ELISA microtiter plate with monoclonal antibody may vary, and this variation may affect the efficacy of binding of galactomannan. Variations in coating conditions may account for the relatively low optical density values of the threshold control samples observed in this study. The interassay agreement was increased by calculating and comparing the ratio for serum, since some factors such as the efficacy of washing of the plates will affect the optical densities of both clinical samples and threshold controls. Nevertheless, the level of variation that we observed in this study suggests that changes in the antigen titer should be interpreted with caution unless consecutive serum samples are analyzed in a single run.