Evaluation of Analytical and Preliminary Clinical Performance of Myconostica MycAssay Aspergillus When Testing Serum Specimens for Diagnosis of Invasive Aspergillosis

ABSTRACT Diagnosis of invasive aspergillosis remains a significant problem. PCR testing may aid diagnosis but is not yet included in disease-defining criteria due to a lack of standardization of assays and methodologies. This study investigated the analytical performance and the clinical sensitivity and specificity of the Myconostica MycAssay Aspergillus PCR (MAP) assay compared to those of a validated in-house Aspergillus PCR (IHP) test when testing serum specimens. Serum specimens spiked with Aspergillus genomic DNA had a limit of detection equivalent to 5 genomes and a linear dynamic range of 5 to >5 × 104 genomes for both assays. When testing clinical specimens (n = 170), the MAP assay had a sensitivity of 60 to 70% and a specificity of 90.5 to 100%. The IHP assay had a sensitivity of 50 to 80% and a specificity of 100%. A commercially available Aspergillus PCR assay provides a methodology that is standardized and reagents that are quality controlled. This facilitates multicenter evaluation of the clinical utility of PCR diagnosis. The performance of the MAP assay is comparable to that of the IHP assay and to those in previously reported studies evaluating commercial tests (galactomannan enzyme-linked immunosorbent assay).

Despite improved awareness, invasive aspergillosis (IA) is still a significant problem in the hematological population, where mortality rates are exacerbated by delayed diagnosis. Nonculture diagnostic techniques (PCR, galactomannan enzyme-linked immunosorbent assay [GM-ELISA], and high-resolution computer tomography) may improve the time to diagnosis and, consequently, reduce mortality rates (8,18,22). Despite showing variable clinical performance, the GM-ELISA is included in the revised European Organization for Research and Treatment of Cancer (EORTC)/Mycoses Study Group guidelines for defining invasive fungal disease (IFD) on the basis of multicenter performance evaluation of a standardized commercial method, whereas PCR is currently excluded through a lack of standardization (4,17).
Only recently has it become possible to purchase commercial Aspergillus PCR assays, and the range of available products remains severely limited compared to the in-house methods. The fact that these in-house protocols have been in use for almost 2 decades but have yet to gain widespread acceptance is a direct consequence of a lack of standardization and multicenter evaluation. The availability of a commercial PCR provides standardized methodology and improved stringency with regard to all aspects of quality control and manufacture and is a step toward incorporating PCR into diagnostic guidelines (5). Multicenter evaluation of clinical performance is essential and can be achieved by independent testing by individual centers, as was the case for GM-ELISA.
Aspergillus PCR has been used to test a variety of specimens (bronchoalveolar lavage [BAL] fluid/sputum, serum/plasma, whole blood, and tissue) with various degrees of success, although direct comparison of specimen type and performance is limited (22). Performance aside, the specimen choice may be directly influenced by the patient's condition, with invasive specimens (BAL fluid, tissue) infrequently being taken. Additionally, the requirements of a screening regime suited to the low incidence of IA necessitate a high frequency of sampling (two to three times/week). Testing of blood samples allows screening regimes to be implemented, and neutropenic care pathways utilizing PCR and ELISA have been shown to have utility in diagnosing and excluding IA (1). A definitive answer as to whether whole blood or serum/plasma is the optimal specimen is yet to be agreed to, although it is accepted that the use of serum is far less technically demanding, targeting free circulating DNA (21,22). Commercial DNA extraction kits can be utilized, saving both time and labor, and have the added benefit of providing further standardization, factors that are beneficial if Aspergillus PCR is to attain widespread use as a diagnostic test.
In this study, the commercially available MycAssay Aspergillus PCR (MAP; Myconostica Ltd., Manchester, United Kingdom) was combined with the Roche High Pure Template DNA extraction kit (Roche, Burgess Hill, United Kingdom) so that it could be used to test serum specimens for the presence of Aspergillus DNA. Its analytical and clinical performance was compared to that of a well-validated in-house Aspergillus PCR test (21,23).

MATERIALS AND METHODS
In reporting this research, the Minimum Information for Publication of Quantitative Real-Time PCR Experiments (MIQE) guidelines have been incorporated, when appropriate, into a qualitative PCR assay (2).

DNA extraction from fungal cultures.
Cultures of Aspergillus fumigatus ATCC 1022 were allowed to sporulate on Sabouraud agar plates before conidia were harvested using a wet sterile loop. The conidia that adhered to the loop were resuspended in sterile water containing a drop of Tween 20 (Sigma, United Kingdom) to prevent clumping. The A. fumigatus suspension was serially diluted to allow accurate quantification using a Fuchs Rosenthal hemocytometer. A volume of suspension containing 10 6 conidia was pelleted by centrifugation (10,000 ϫ g for 5 min), and the supernatant was discarded. The pellet was exposed to mechanical disruption with the equivalent of 20 to 30 l of MagNA Lyser Green beads (Roche, United Kingdom) and 30 s bead beating using a minibeadbeater (Biospec Products). After pulse centrifugation, the beads were washed with 200 l molecular-grade water and DNA was extracted using the Roche High Pure Template DNA kit, per the manufacturer's instructions, using an elution volume of 65 l.
DNA extraction from serum specimens. As the Myconostica MycXtra kit is specifically designed for testing BAL fluid specimens, DNA was extracted from 0.5 ml of serum using the Roche High Pure Template DNA kit. All reagents were filter sterilized prior to use. To account for the larger input sample volume, initial reagent volumes were doubled, resulting in a specimen mix requiring two applications to the spin column. Briefly, 0.5 ml of serum was mixed with 0.4 ml of binding buffer and 80 l of recombinant proteinase K (Roche), and the mixture was incubated at 70°C for 10 min. Two hundred microliters of isopropanol was added and mixed by pipette before the entire specimen (by repeat addition) was applied to the spin column by centrifugation at 8,000 ϫ g for 1 min. Other than these modifications, the kit was used as per the manufacturer's instructions, using an elution volume of 65 l.
PCR. For all molecular assays, measures to control contamination and maintain performance were strictly followed (11). PCR efficiency was calculated using the following equation: Ϫ1 ϩ 10 (Ϫ1/slope) (2).
(i) IHP test. The in-house Aspergillus real-time PCR (IHP) test was performed as a single-round assay using a Roche LightCycler (version 2.0) instrument targeting the 28S rRNA gene as previously described, with 10 l DNA input in a final reaction volume of 20 l (23). In addition to extraction controls, PCR controls in the form of cloned PCR products (200, 20, and 2 input copies) and no-template molecular-grade water were included to monitor PCR performance. PCR positivity was determined using a threshold of 45 cycles.
(ii) IC PCR. The in-house internal control (IC) PCR targeted the capsular transfer (ctrA) gene of Neisseria meningitidis. It was performed on the Roche LightCycler (version 2.0) instrument, using the monoplex version of the assay, as described by Corless et al. (3), using a 10-l DNA input in a final reaction volume of 20 l (20). PCR controls in the form of the plasmid containing the ctrA gene incorporated into the extraction procedure and no-template molecular-grade water were included to monitor PCR performance.
(iii) MAP. MAP was performed on a Cepheid SmartCycler instrument as per the manufacturer's instructions using 10 l of DNA template in a final reaction volume of 25 l and targeted the 18S rRNA gene. The kit contained positive and no-template controls along with the master mix containing reagents for the Aspergillus PCR and IC PCR, which was run as a duplex real-time PCR utilizing fluorescently labeled beacons. PCR positivity was determined using a threshold of 39 cycles.
Verification of analytical performance. To establish the detection range, limit of detection, limit of blank, and reproducibility of the MAP assay when testing serum, separate investigations were performed by the manufacturer and independent testing center. Analytical specificity has been previously reported for both Aspergillus PCR tests, and this will not change when testing a different specimen type (6,23). The analytical performance of the assays when testing simulated serum specimens was determined prior to testing of clinical samples.
Detection range and LoD. A. fumigatus genomic DNA previously extracted from known conidial quantities was diluted and used to spike serum samples with various burdens before the DNA was extracted and the MAP assay was performed. In addition, the independent center also tested the DNA extracts by the IHP and IC PCR. Quantification was determined on the basis of one A. fumigatus conidium containing one genome and the mean number of rRNA copies per genome being 53 (9). Comparative evaluation of MAP and IHP performance in detecting eight Aspergillus species (A. fumigatus, A. flavus, A. terreus, A. niger, A. nidulans, A. versicolor, A. sclerotiorum, and A. glaucus) at a single burden was determined. Reproducibility at the limit of detection (LoD) was determined by the independent center for both assays by replicate testing during a single experiment (n ϭ 5, intra-assay) and over multiple experiments (n ϭ 10, interassay).
Determination of clinical performance. A pilot study was performed on a group of 31 high-risk hematology patients. The study took the form of a retrospective anonymous service evaluation of hematology patients. As part of the routine neutropenic fever care pathway, clotted blood specimens were sent twice weekly for IA screening by PCR and GM-ELISA (1). After initial diagnostic investigations, serum samples were stored at Ϫ80°C for future internal quality assessment procedures and to permit service evaluation investigations. IFD was defined according to the revised EORTC criteria (4). Ten cases were selected for testing on the basis of having proven/probable IA, as defined by positive histopathology and/or defined radiological signs plus GM-ELISA, and 21 patients with no evidence of IA were used as a control population. The mean patient age was 48.1 years, with an age range of 23 to 84 years. Of the 21 patients with no evidence of IA, 7 were diagnosed with a respiratory virus infection and 3 with cytomegalovirus infection, 1 had an Escherichia coli bacteremia, and 2 were diagnosed with non-Aspergillus mold infections (1 with proven mucormycosis and 1 patient with Fusarium dimerum cultured from a respiratory specimen). The remaining patients had no documented cause for their neutropenic fever.
For the purpose of this study, 0.5 ml of serum from patients with and without IA was processed as described above. A total of 170 specimens (104 specimens from 10 cases of proven/probable IA [median, 11; range, 4 to 22] and 66 from patients at high risk of IA [median, 2; range, 1 to 22]) were retrospectively tested by serum PCR.
DNA extracted from culture was serially diluted to obtain DNA concentrations equivalent to ranges experienced in the clinical scenario (5 to 25 genome equivalents [ge]) and added to serum for positive-control material. Negative controls in the form of serum from a healthy donor were also used. To monitor for sample inhibition and extraction efficiency, an IC was included in each specimen.

RESULTS
Detection range. Both real-time assays were tested over a 5-log 10 linear range of 5 to 5 ϫ 10 4 ge per simulated serum sample, which corresponds to 41 to Ͼ 4 ϫ 10 5 rRNA input copies per PCR (Table 1 and Fig. 1). Both assays were able to detect a range of Aspergillus species, although crossing point (Cq) values were earlier for the MAP assay, and the IHP detected A. glaucus only at a later stage of amplification (Table  2). When detecting 5,000 copies, the Cq value for the MycAssay Aspergillus was, on average, 5.95 cycles (range, 2.5 to 15.7 cycles) earlier than the Cq for the in-house PCR, and this was more evident when testing DNA from non-A. fumigatus species. One of 18 negative healthy donor serum samples, when tested at the independent center, was positive by the IHP and MAP assays on separate occasions. Retesting of these extracts failed to confirm the false positivity, indicating that operatorassociated PCR setup contaminations were responsible for the initial false positivity.
PCR efficiency, LoD, and reproducibility when testing serum. The PCR efficiencies for the MAP and IHP assays were calculated from the slope of each standard curve and were 1.01 and 0.90, respectively (Fig. 1). Cq values were earlier for the MAP than the IHP (Table 1), although the assays used different positivity thresholds. All extracts were easily detected within the defined positivity ranges of the assays. The manufacturer's 100% LoD for the MAP assay was 5 ge per sample (16). For the independent center, an initial 100% LoD of 50 ge per sample was determined for the MAP assay, with reproducibility for 5 ge being 80% (Table 1). This discrepancy was attributed to the use of A. fumigatus genomic DNA that had been subjected to long-term storage (Ͼ6 months). Analysis of results when using only freshly extracted DNA improved the 100% LoD for the MAP assay at the independent center to 5 ge per sample, generating a mean Cq value of 36.7 (standard deviation [SD], Ϯ0.5; 95% confidence interval [CI], 36.2 to 37.1) cycles, confirming the manufacturer's findings. The 100% LoD for the IHP determined by the independent center was 50 ge per sample, but again, removal of the data generated by the testing of stored extracts improved this to 5 ge, generating a mean Cq value of 38.3 (SD, Ϯ1.2; 95% CI, 37.1 to 39.4) cycles. The interassay (between-run) mean Cq value for the MAP assay at the LoD was 36.8 (n ϭ 10; SD, Ϯ0.6; 95% CI, 36.3 to 37.3) cycles. The intra-assay (within-run) mean Cq of the MAP assay at the LoD was 36.4 (n ϭ 5; SD, Ϯ0.42; 95% CI, 35.9 to 36.9) cycles. The interassay mean Cq value for the IHP assay at the LoD was 38.6 (n ϭ 10; SD, Ϯ1.4; 95% CI, 37.4 to 39.7) cycles. The intra-assay mean Cq of the IHP assay at the LoD was 37.9 (n ϭ 5; SD, Ϯ0.61; 95% CI, 37.1 to 38.6) cycles.
Determination of clinical performance. The diagnostic performance parameters when testing 104 serum specimens from 10 patients with proven/probable IA and 66 serum specimens from 21 patients at risk from IA by both real-time PCR assays are shown in Table 3. Seven of the 10 patients with proven/ probable IA were MAP assay positive on at least one occasion, and 6 had multiple (Ն2) MAP-positive results. Eight of the patients with proven/probable IA were IHP assay positive on at least one occasion, and five had multiple (Ն2) IHP-positive results. Twenty of the 104 specimens from patients with proven/probable IA were PCR positive when tested by the MAP and IHP assays, with 13 MAP-positive and 12 IHP PCR-positive results being generated when testing samples taken within 1 week of clinical diagnosis and 71.4% and 75% of patients with IA being MAP and IHP PCR positive, respectively, prior to clinical signs.
Both assays are highly specific, with specificity values always being Ͼ90%, and patients with multiple positive PCR results were always associated with disease, as indicated by the likelihood ratio positive ( Table 3). The diagnostic odds ratios were typical of those of Aspergillus PCR assays used to test blood Individual specimen concordance between serum PCR assays was good ( Table 4). The accuracy between the two serum PCR tests when testing serum from patients with proven/probable IA was 88.5% (95% CI, 80.6 to 96.3%), generating a kappa statistic of 0.63 (95% CI, 0.44 to 0.82). For the specimens from patients without disease, the accuracy between the two serum PCR tests was 97.0% (95% CI, 92.9 to 100%). The overall accuracy between the two serum PCR tests was 91.8% (95% CI, 86.5 to 97.0%), generating a kappa statistic of 0.62 (95% CI, 0.47 to 0.77), representing good agreement (Table 4).

DISCUSSION
An evaluation has been conducted to assess the performance of a commercially available, and therefore standardized, Aspergillus PCR assay when testing serum. The performance of the MycAssay Aspergillus assay was comparable to that of a wellvalidated in-house PCR test, and both assays evaluated in this study generated acceptable sensitivity values comparable to previously published values for Aspergillus serum PCR assays that generated pooled sensitivities of 72% (range, 53 to 100%; 95% CI, 66 to 77%) (22).
This was a retrospective serum PCR study; therefore, it is relevant to consider the effects of storage and DNA degradation. When verifying the manufacturer's LoD, initial testing of stored DNA resulted in the reproducibility of detection being reduced compared to that obtained by testing of freshly extracted DNA by values of 33.3% (95% CI, 3.9 to 70.0%) and 66.7% (95% CI, 6.5 to 93.9%) for burdens of 25 and 5 ge, respectively. Many of the clinical specimens were stored for a similar period (Ͼ6 months), and it is therefore reasonable to assume that some degradation will have occurred. As A. fumigatus DNA has been shown to be stable in serum and whole blood for relatively short periods of time (Յ144 h), it is likely that a prospective study will improve detection rates (15).
Assay specificity was excellent, with all assays achieving Ͼ90% specificity irrespective of the definition of PCR positivity, indicating the importance of a single PCR-positive specimen, particularly when PCR positivity for blood samples may be as low as 11% in patients with proven/probable IA (19). However, in a previous review and meta-analysis of PCR used to test blood specimens, a single PCR-positive result was as- a The specificity panel was developed by the commercial partner using DNA extracted from culture and donated to Myconostica by multiple independent centers. DNA concentrations were measured and used to calculate numbers of copies on the basis of 1 fg of genomic DNA containing 1 copy of the 18S rRNA gene (10). Both PCR tests were performed in duplicate for each species. Five thousand copies is equivalent to approximately 10 2 genomes (conidia), assuming the mean number of rRNA genes of A. fumigatus to be 53 per genome (9). As the number of copies of rRNA target will vary between Aspergillus species, the DNA input has been provided as an approximate comparative guide and not for quantitative purposes. In-house Aspergillus PCR  sociated with reduced specificity (13). In this study, PCR sample positivity when testing serum from patients with proven/ probable IA was higher (19.2%) but ranged from 0 to 100% between individual patients, possibly representing sample degradation. Low sample positivity rates may be a result of minimal circulating target, and the mean Cq values for PCR-positive specimens provide mean fungal burdens of 11.6 and 10.1 ge/sample for the IHP and MAP assays, respectively. A low circulating burden may be a consequence of testing blood specimens to monitor for a disease that in its initial stages will be focused in the respiratory tract, and for saprophytic diseases (i.e., aspergilloma), assay sensitivity has been shown to be compromised (24). Invasive disease progresses through tissue and angioinvasion, providing a target in the bloodstream (fungal cell-associated and free DNA [see below]) that may be enhanced by the detection of conidia phagocytosed in alveolar macrophages that have been translocated into the circulation, a pathway that has been determined using macrophages containing fluorescent particles (7). Nevertheless, as a true fungemia is unlikely with IA, the burden in blood will be low and transient, requiring frequent screening to provide sensitivity. A conclusive comparison of the performance of Aspergillus PCR when testing serum/plasma or whole blood has yet to be published. Technically, serum testing has many advantages over whole-blood testing for Aspergillus PCR. Both PCR and GM-ELISA can be performed on a single specimen, allowing direct test comparison and providing cost and patient benefits. Testing serum targets free circulating DNA, as any cell-associated fungal DNA will be lost during the processing necessary to fractionate the blood. In vivo release of DNA most likely occurs through hyphal damage by autolysis or immune-mediated damage. Many antifungal therapies target the fungal cell wall or membrane, causing structural damage and DNA release, possibly enhancing detection (14). Targeting free circulating DNA permits the use of simple commercial, possibly automated, DNA extraction techniques, both standardizing and improving turnaround time for Aspergillus PCR. The IC target (usually plasmid DNA) is also more relevant for serum than for whole blood, where it is likely to be destroyed or decanted during the aggressive extraction procedure and can be incorporated only in the later stages of the process. For serum testing, this is not the case, and the incorporation of an IC at the start of the extraction process monitors for both inhibition and extraction efficiency on an individual sample basis, eradicating the need for a positive extraction control, thus removing a potential source of contamination. It must be noted that currently the internal control for the MAP assay is included in the PCR master mix and so can be used only to monitor for PCR inhibition, as such a positive extraction control is required.
The retrospective performances of both serum PCR tests highlight the importance of a prospective comparative study of GM-ELISA, whole-blood Aspergillus PCR, and serum PCR in order to determine an optimal specimen type and diagnostic strategy. Although to provide a true comparison with GM-ELISA it will be necessary to modify disease-defining criteria or provide an evaluation of assay performance when testing patients with histopathologically proven disease or cases of probable IA defined by radiological evidence plus positive culture. Nevertheless, this retrospective evaluation of serum PCR still generates performance statistics comparable to those of GM-ELISA, which generated pooled sensitivities of 71% (95% CI, 68 to 74%) and 78% (95% CI, 61 to 89%) (12,17).
In conclusion, the aim of this study was to evaluate the performance of an entirely commercially available Aspergillus PCR assay providing a methodology that is standardized and reagents that are quality controlled. The performance of the MAP assay is comparable to that of an in-house PCR method testing serum specimens and other previously evaluated commercial tests (GM-ELISA) to aid in the diagnosis of IA. It is essential that a large-scale evaluation of the MAP assay be performed, and it is likely that a prospective evaluation will improve clinical performance, as target degradation will be less of a concern. By providing a standardized commercial approach, future multicenter performance parameters can be determined without concerns regarding method variability, permitting the clinical validity and utility of PCR to be accurately determined at multiple independent sites.