ABSTRACT
In the case of a release of highly pathogenic bacteria (HPB), there is an urgent need for rapid, accurate, and reliable diagnostics. MALDI-TOF mass spectrometry is a rapid, accurate, and relatively inexpensive technique that is becoming increasingly important in microbiological diagnostics to complement classical microbiology, PCR, and genotyping of HPB. In the present study, the results of a joint exercise with 11 partner institutions from nine European countries are presented. In this exercise, 10 distinct microbial samples, among them five HPB, Bacillus anthracis, Brucella canis, Burkholderia mallei, Burkholderia pseudomallei, and Yersinia pestis, were characterized under blinded conditions. Microbial strains were inactivated by high-dose gamma irradiation before shipment. Preparatory investigations ensured that this type of inactivation induced only subtle spectral changes with negligible influence on the quality of the diagnosis. Furthermore, pilot tests on nonpathogenic strains were systematically conducted to ensure the suitability of sample preparation and to optimize and standardize the workflow for microbial identification. The analysis of the microbial mass spectra was carried out by the individual laboratories on the basis of spectral libraries available on site. All mass spectra were also tested against an in-house HPB library at the Robert Koch Institute (RKI). The averaged identification accuracy was 77% in the first case and improved to >93% when the spectral diagnoses were obtained on the basis of the RKI library. The compilation of complete and comprehensive databases with spectra from a broad strain collection is therefore considered of paramount importance for accurate microbial identification.
INTRODUCTION
Highly pathogenic bacteria (HPB) are risk group 3 bacteria, which are defined as biological agents that can cause severe human disease and present a serious hazard to health care workers. This may present a risk of spreading to the community, but there is usually effective prophylaxis or treatment available (1). To this group belong bacteria such as Bacillus anthracis, Francisella tularensis subsp. tularensis (type A), Yersinia pestis, species of the Brucella melitensis group, Burkholderia mallei, and Burkholderia pseudomallei. HPB have the potential to be used in bioterrorist attacks (2, 3). The Centers for Disease Control and Prevention (CDC, Atlanta, GA) have classified B. anthracis, F. tularensis, and Y. pestis as category A and Brucella species, B. mallei, B. pseudomallei, and Coxiella burnetii as category B, comprising the main pathogens of concern for use in bioterrorist attacks (4). These pathogens may cause anthrax, tularemia, plague, brucellosis, glanders, melioidosis, and Q fever, respectively. In most parts of the world, the natural prevalence of these agents is low, even though some of these agents cause outbreaks in human and animal populations from time to time (5–8). The intentional release of these agents, however, can result in severe public health consequences, as was shown in the Unites States in 2001 (9, 10). Therefore, accurate assays for microbial identification are important to ensure proper medical intervention, either in the case of a natural outbreak or an intentional release. Such assays must be able to identify unambiguously a broad panel of potential threat microorganisms in different background matrices that may or may not be contaminated with non-HPB bacteria (11).
Matrix-assisted laser desorption ionization–time of flight (MALDI-TOF) mass spectrometry (MS) is a rapid, accurate, sensitive, and cost-effective method that offers an adequate alternative to genome-based approaches and that has been widely used for identification and typing of microorganisms in a clinical routine setup (12–19), as well as for HPB (20–27). This method does not depend on exclusive consumables and has revealed high levels of reproducibility in both intralaboratory and interlaboratory studies (28, 29). Whole cells, crude cell lysates, or bacterial extracts can be utilized to generate taxon-specific fingerprint signatures (30). For safety reasons, the application of MALDI-TOF MS for HPB requires complete inactivation of the microbial samples unless the mass spectrometer is operated in a biosafety level 3 (BSL-3) laboratory. As this is often impossible, whole-cell preparations or crude cell lysates cannot be used for MS-based analyses of HPB.
In this paper, we describe an international exercise for identification of HPB by MALDI-TOF MS which was carried out in the framework of the European Union (EU)-funded project Quality Assurance Exercises and Networking on the Detection of Highly Infectious Pathogens (QUANDHIP). The aim of this Joint Action (JA) was to build up a stabilized consortium that links up 37 highly specialized laboratories from 22 European countries and to guarantee universal exchange of the best diagnostic strategies to support a joint European response to outbreaks of highly pathogenic infectious agents. The JA provided a supportive European infrastructure and strategy for external quality assurance exercises (EQAE), training, and biosafety/biosecurity quality management. The aim of this EQAE was (i) to evaluate the current state of the MALDI-TOF MS-based identification technique for highly pathogenic agents in Europe, (ii) to explore opportunities to advance the diagnostic capabilities, including optimization and standardization of the diagnostic workflow and exchange of standards and protocols (e.g., for verification of MS-compatible inactivation methods), and (iii) to implement measures to improve MALDI-TOF MS-based diagnostics of HPB in Europe (capacity building). The exercise was conducted as a blinded interlaboratory study with 10 different bacterial isolates representing five HPB and five non-HPB test strains and involved in the preparatory-phase pilot tests on non-HPB and inactivation tests with γ-irradiated microorganisms. Eleven QUANDHIP project partners from nine European countries participated in this exercise, including three laboratories from Germany and one each from Austria, the Czech Republic, Denmark, Hungary, Italy, Norway, Sweden, and Switzerland.
MATERIALS AND METHODS
Microbial strains and isolates.All microbial strains originated from the international QUANDHIP strain collection deposited at the Highly Pathogenic Microorganisms Unit (ZBS 2) at the Robert Koch Institute (RKI) in Berlin. These strains represent mainly patient isolates sent by the participating laboratories to the QUANDHIP strain collection. All strains were characterized twice, first in the laboratories that provided the strains and second at RKI/ZBS 2 by means of a large variety of different methods, including classical microbiological, PCR-based, and genotyping methods. An overview of the strains and isolates used in this study is given in Table 1. All microbial strains and isolates were handled according to the respective biosafety regulations outlined in Technical Rule for Biological Agents 100 (TRBA 100: Protective Measures for Activities Involving Biological Agents in Laboratories) (31). HPB and F. tularensis subsp. holarctica (type B; risk group 2), a very close relative of F. tularensis subsp. tularensis (type A; risk group 3), were handled according to the requirements of TRBA-100 in a BSL-3 laboratory. The strains were grown under optimal aerobic or microaerophilic conditions on Columbia blood agar plates from Oxoid, Wesel, Germany (Bacillus spp., Yersinia spp., Burkholderia spp., Brucella spp., and Ochrobactrum spp.) or on heart cysteine agar (HCA) plates (Francisella spp.) for at least 24 h and up to 72 h at 37°C. HCA agar plates were produced in house from an agar base obtained from Bestbion dx (Cologne, Germany) and sheep blood (Oxoid). Except for Francisella species isolates, all strains were transferred once onto tryptic soy agar (TSA; VWR, Darmstadt, Germany) or Caso agar (Merck KGaA, Darmstadt, Germany). Cells were harvested from the second passage by resuspending colonies in double-distilled water to an optical density at 600 nm of between 1.0 and 1.2.
Overview of microbial strains and species used in the interlaboratory ring trial
Sample preparation/sample inactivation.The concentration of CFU in the microbial suspensions was adjusted to between 107 and 1010 CFU per ml (Table 1). The suspensions were stored at −75°C until further treatment. Inactivation of microbial samples was carried out by means of high-dose gamma irradiation. For this purpose, microbial suspensions were sent on dry ice from the RKI to Synergy Health Radeberg GmbH (Radeberg/Germany) in accordance with the Dangerous Goods Regulations for category A organisms with UN 2814 (60). Irradiation was carried out according to the recommendations of ISO 11137 (61) using a 60Co gamma ray source. The measured irradiation dose varied between 27.34 and 32.68 kGy. To minimize the possible radiation-associated spectral changes (thermal degradation), the samples were transported and irradiated in the frozen state. For this purpose, all samples were shipped along with a large amount of dry ice. After sample return, it could be verified that a sufficient amount of dry ice was still present and that the samples were not thawed at any time. Tests for sterility after irradiation were conducted by cultivation. In these tests, 10% (vol) of each overall sample solution was added to in-house-produced tryptic soy broth (TSB; Oxoid, Wesel, Germany). Additionally, 100 μl of each sample was twice plated onto the appropriate medium, usually Columbia blood agar or HCA plates (Francisella). Incubation for growth in TSB was carried out over a time span of 14 days. Final culturing was performed on Columbia blood agar or HCA plates (Francisella) if visible turbidity of TSB was not observable. All agar plates were incubated under species-specific ideal conditions for 3 to 7 days. For the EQAE, only samples which showed no growth after gamma irradiation, either in TSB or on Columbia blood agar or HCA plates, were used.
The inactivated microbial samples were aliquoted (1 ml) and stored again at −75°C until shipment. The aliquots were shipped to the 11 partner institutions on dry ice. Before shipment, blinded MALDI-TOF MS test measurements were performed at the Proteomics and Spectroscopy Unit (ZBS 6) to assess the suitability for MALDI-TOF MS.
When setting up their own spectral databases prior to the ring trial, all partners could choose among a large variety of procedures, protocols, and parameters of sample preparation and data acquisition. While some participants routinely utilize the so-called direct transfer method (30, 32) and/or the ethanol-formic acid (FA) protocol recommended by Bruker Daltonics (30, 33), the group at RKI primarily uses the trifluoroacetic acid (TFA) inactivation/sample preparation method (34). A large advantage of inactivation by gamma irradiation is that this method is compatible with all of these sample preparation protocols: microbial isolates inactivated by gamma irradiation can in principle be further processed by utilizing any of the different laboratory-specific methodologies. This allowed optimal usage of in-house spectral databases compiled by the individual partner institutions prior to the ring trial. The specific details and settings of the various experimental protocols were polled as a substantial part of the preparation of the ring trial and are summarized in Table S1 in the supplemental material. Furthermore, the preparation of the exercise included systematic MS pilot tests of non-HPB strains by each participating institution. These tests were performed with the aims (i) of identifying and eliminating possible sources of underperformance, such as inadequate procedures of sample preparation or poor parameter selection, and (ii) standardizing, wherever possible, experimental procedures and data acquisition protocols. Within the scope of these pilot tests, MALDI-TOF mass spectra of Bacillus thuringiensis, Burkholderia thailandensis, Escherichia coli, and Yersinia enterocolitica were acquired, shared, and jointly analyzed.
MALDI-TOF MS.Details of MALDI-TOF MS measurements are in Table S1 in the supplemental material.
Identification approach A.The analysis of mass spectra from blinded microbial samples was carried out first on-site by the ring trial participants themselves. In this approach, the participants employed different types of identification software and utilized a variety of distinct mass spectral libraries, such as Bruker's commercial database for clinical microbiology, the standard BioTyper database, the so-called Security Relevant reference library (SR library) from Bruker, the SARAMIS database, and in-house databases compiled by the institutions themselves (see Table S1 in the supplemental material). During EQAE's preparatory stage, some of the ring trial participants initiated data exchange activities with the purpose of increasing the size and improving the degree of coverage of these in-house libraries.
Identification approach B.After submission of the identification results, all mass spectra were collected at the study center (RKI) and subsequently analyzed for a second time using the database of HPB at RKI. This in-house database consists of 1,118 entries (main spectral projections [MSPs]), each corresponding to a defined microbial strain from the genera Bacillus, Burkholderia, Brucella, Francisella, Vibrio, and Yersinia (along with a number of clinically relevant species from the genera Escherichia, Enterococcus, Staphylococcus, Streptococcus, and others). These MSPs represent database entries in the server component of Bruker's BioTyper software package which can be queried via BioTyper software clients (version 3.1, build 66; Bruker). Microbial identification was achieved on the basis of the unmodified standard BioTyper identification method compiled by the manufacturer. Furthermore, identification was conducted by means of logarithmic scores with cutoff values as specified by Bruker: log score values larger than 2.3 are required for a reliable (highly probable) identification on the species level, and scores of between 2.3 and 2.0 represent probable species identification. Scores of between 2.0 and 1.7 point toward a reliable genus identification, while values below 1.7 are regarded as unreliable (35). Due to the proprietary nature of the spectral data file format, analysis in identification approach B was limited to spectra acquired by mass spectrometers produced by Bruker: the BioTyper client software does not allow importing data in a format other than the Bruker format. As one of the participating institutions employs MS equipment produced by Shimadzu (laboratory XI), identification approach B involved the analyses of MS data from 10 of the 11 participating institutions.
Identification approach C.In the third analysis approach, the Matlab-based software solution MicrobeMS version 0.72 (24, 36–39) developed at RKI was used (Matlab; The Mathworks, Inc., Natick, MA). MicrobeMS is publicly available as Matlab p-code (free of charge) and provides direct access to Bruker's raw spectral data and to spectra acquired by the Vitek MS workflow (formerly SARAMIS, bioMérieux/Shimadzu) via the mzXML data format (40). The software allows spectral preprocessing, such as smoothing, baseline correction, intensity normalization, and internal calibration, and can be employed to produce reference peak lists from microbial MALDI-TOF mass spectra (39). Furthermore, MicrobeMS can be used to systematically screen for taxon-specific biomarkers and for visualization of large spectral data sets (via pseudogel views). Within the context of the present study, the software was utilized for identification purposes in combination with the mass spectral database for HPB. This allowed cross-platform analysis of microbial mass spectra from partner institutions using instrumentation from two different manufacturers, Bruker and bioMérieux/Shimadzu (see reference 39 for details).
RESULTS AND DISCUSSION
Gamma inactivation.Complete inactivation of all pathogens prior to dispatch to the ring trial participants was considered an essential prerequisite for successful implementation of the interlaboratory ring trial. Although it would in principle have been possible to distribute viable BSL-3 pathogens throughout Europe, the very strict legal provisions would have represented a significant organizational challenge with very high shipment costs. The shipment of viable BSL-3 samples is only allowed as infectious material (class 6.2) category A in accordance with the Dangerous Goods Regulations, whereas inactivated material can be dispatched very easily.
As stated earlier, gamma irradiation was selected as the inactivation method of choice. Although the TFA sample preparation protocol has been specifically developed as a MALDI-TOF MS-compatible method for microbial inactivation of HPB, it was decided not to employ this protocol. It is well-known that spectra produced by acid-based methods exhibit systematic changes compared to spectra created by the direct transfer method (41). Differences between spectra obtained by the ethanol-FA and the TFA method, however, are much smaller, since both techniques are ultimately based on acid extraction. In any case, shipment of γ-inactivated biological material allowed the partners to choose the appropriate preparation protocol, which resulted in a very high degree of compatibility with existing in-house database solutions at the partner institutions.
High-dose gamma irradiation is known in the literature as a method suitable for reliably inactivating bacterial pathogens (42, 43) while leaving the primary protein structures basically intact. Our comparative measurements of pathogenic and nonpathogenic microbial strains essentially confirmed the data in the literature: identification was successful after high-dose gamma irradiation, but irradiation resulted in slightly lower BioTyper log score values (data not shown). Under the specific experimental conditions at RKI, it was found that the signals relevant for identification remained very marked after gamma irradiation, though with reduced peak intensities. The MALDI-TOF mass spectra of E. coli and B. cereus exemplarily demonstrate the presence of all of the main peaks in both the irradiated and the reference samples (Fig. 1). However, the spectra of the γ-inactivated samples generally exhibited a lower signal-to-noise ratio due to the slightly reduced peak intensities.
MALDI-TOF mass spectra of control samples (black traces) and microorganisms inactivated by means of high-dose gamma irradiation (red traces). Irradiated samples of E. coli A139 and B. cereus BW-B (spores) were prepared for MALDI-TOF MS in the same way as the retained control samples, by means of the TFA inactivation method (34). The spectra (smoothed and baseline corrected) demonstrate only insignificant differences between the irradiated and control samples, suggesting that gamma irradiation is compatible with the routine sample preparation protocols used by the partner institutions (see the text for details). Intens. [a.u.], intensity in arbitrary units.
Pilot tests on non-HPB strains.Pilot tests were conducted by the partners to identify factors that affect the overall performance of the MS-based identification technique and to standardize experimental procedures, data acquisition protocols, and methods of spectral analysis. In the context of the preparation of the pilot tests, experimental methods and parameters were polled (see Table S1 in the supplemental material).
The jointly conducted analysis of microbial MALDI-TOF mass spectra from non-HPB revealed a number of peculiarities, such as broadened peaks, spectrum baseline irregularities (elevated baselines), and the appearance of additional satellite peaks in some of the microbial mass spectra. While peak broadening and baseline elevation effects could be identified relatively easily as a result of the application of excessive laser power (see reference 44), it was more challenging to identify the sources and causes of additional satellite peaks.
Satellite peaks.In Fig. 2, the bottom panel illustrates a first example of satellite peaks in a mass spectrum of B. thuringiensis. As shown, additional peaks such as are described above occurred at 16-Da intervals at higher molecular masses with respect to the parent peak (cf. peak series at m/z 4,335, 4,351, and 4,367). The spectrum of B. thuringiensis obtained by the reference sample preparation method (TFA inactivation) clearly demonstrates the absence of such peaks in the control measurements (Fig. 2, top). The observed satellite peaks are most likely caused by the action of sodium hypochlorite (NaClO), which is known as an effective disinfectant and a strong oxidizing agent. Because of its well-known antimicrobial and sporicidal properties, NaClO was applied in the laboratory of one of the partners for 15 min in a concentration of 10% (vol/vol) for external sterilization of the MALDI-TOF MS sample vials. It seems likely that during this period, small amounts of NaClO entered the tubes, e.g., via incompletely closed lids. In proteins, the amino acid methionine and aromatic residues like tryptophan and tyrosine are potential first oxidation targets (45, 46). In the case of oxidation of methionine, the experimentally observed mass differences between the parent and satellite peaks of 16 Da would fit well with the computed masses of unoxidized methionine and methionine sulfoxide as the singly oxidized species (47). However, the mass differences mentioned would be also observable in the case of oxidation of other amino acids.
Oxidation of Bacillus thuringiensis microbial extracts by sodium hypochlorite (NaClO). Top, reference mass spectrum of a B. thuringiensis sample prepared on the basis of the trifluoroacetic acid (TFA) inactivation technique (34). Bottom, TFA-treated sample of the same Bacillus strain with likely contamination by sodium hypochlorite. The spectral differences—satellite peaks at +16-Da intervals—are attributed to contamination by the oxidant NaClO, which was employed for external sterilization of sample vials during outward transfer from a BSL-3 laboratory (spectra were smoothed and baseline corrected; see the text for further details).
Similar oxidation-induced satellite peaks (Δm/z of +16 Da) were observed when microbial samples were accidentally inoculated with another sterilizing agent, peracetic acid (PAA). PAA also acts as an oxidizing agent and can cause the oxidation of lipids and amino acid side chains of peptides and small proteins in microbial extracts (data not shown).
Satellite peaks were also detected in samples prepared by means of the ethanol-FA sample preparation protocol (30, 33). Using the examples of spectra from B. cereus ATCC 10987 and B. thuringiensis DSM 5815, the traces in Fig. 3 show the presence of additional peaks at a distance of 28 Da: the black curves denote mass spectra in the m/z 6,250-to-7,500 region of Bacillus samples prepared by the TFA inactivation method, while the red spectra were obtained from identical Bacillus strains prepared by means of the ethanol-FA sample preparation method, which included incubation in 70% FA (vol/vol) for 30 min. Both pairs of spectra display parent peaks at m/z 6,695 (B. cereus) or m/z 6,711 (B. thuringiensis), assigned as a beta-type small, acid-soluble protein (β-SASP), m/z 6,835 (α-SASP), and m/z 7,082 (α-β-SASP) (see references 24 and 48 for peak assignments). Apart from these dominating signals, the spectra of FA-treated samples exhibit additional satellite peaks at m/z 6,723 (B. cereus) or m/z 6,739 (B. thuringiensis) and at m/z 6,863 (both strains). Satellite signals are found at a distance of +28 Da from the parent peaks, typically with intensities of less than 20% of the original signal. A likely explanation for the occurrence of satellite peaks would be chemical modification of the SASPs (formyl esterification) due to prolonged sample treatment by FA. FA treatment has been associated with formylation of proteins in microbial extracts (49), with the specific targets of serine and threonine residues. Furthermore, it is known that formylation is particularly effective when highly concentrated FA is applied to small hydrophobic proteins (50), such as SASPs. Since each additional satellite peak may potentially have a negative impact on the performance of the identification algorithm, the exposure time to FA should be minimized whenever possible. Taking into account that this note is also given in the BioTyper manual (see reference 35), the reduction of FA incubation time is considered an important measure for improving the accuracy of identification.
Formylation of spore marker proteins (small, acid-soluble proteins [SASPs]) in test samples of Bacillus cereus and Bacillus thuringiensis possibly as a result of prolonged treatment by highly concentrated (70%) formic acid (FA) (24). Hash marks (#) indicate peaks at m/z 6,695 or 6,711 corresponding to two possible variants of β-SASP in B. cereus and B. thuringiensis. Ampersands (&) indicate peaks at m/z 6,835 (α-SASP in B. cereus ATCC 10987; UniProt Q73CW6). All mass spectra were smoothed, baseline corrected, and intensity normalized. Black traces are reference MALDI-TOF mass spectra of Bacillus samples prepared by the TFA inactivation method (34). Red traces are spectra from identical strains processed by the ethanol-FA method (33). Red numbers denote additional mass peaks at a distance of +28 Da with reference to the α-SASP (m/z 6,835) or the β-SASP (m/z 6,695/6,711) peaks.
Results of the interlaboratory ring trial.Table S2 in the supplemental material has a summary of the identification results in the context of identification approach A. This approach involved data analysis on site by each partner institution. The table not only shows an overview of the results of the blinded identity tests, it also provides either the logarithmic BioTyper scores or, alternatively, the respective SARAMIS score values. In approach A, MALDI-TOF mass spectra acquired by laboratory XI were analyzed twice, first by using customized in-house algorithms and second by an analysis carried out elsewhere by means of the SARAMIS software and the database solution from Anagnostec. For this reason, Table S2 includes an additional column headed “Laboratory XII,” which is different from identification approaches B and C.
The color scheme used in Table S2 is a traffic light coloring scheme: it uses the colors green for correct, yellow for partially correct, and red for false identification results. A correct result was obtained when the identity was accurately revealed at the genus, species, and subspecies levels. Cells colored yellow denote identification results which were either incomplete—for example, in cases where the subspecies specification was lacking (see sample 2, F. tularensis subsp. holarctica, in Table S2)—or where the genus assignment was correct but the species was left unassigned (e.g., in lines 9 and 10, laboratory VIII, Yersinia sp. and Bacillus sp., in Table S2). Furthermore, a result was also considered partially correct in cases of contradictory identification results, i.e., if different microbial identities were obtained from spectra of technical replicate measurements. In such cases, however, at least one result had to be correct. An example of contradictory identification results can be found in Table S3 for sample 6 from laboratory X. Score values in this or similar instances were indicated by a range of values. Identification results were considered incorrect (red cells) if either an HPB was clearly assigned as a non-HPB (false negative) or, alternatively, if a non-HBP was identified as an HPB (false positive). Cases where no false-positive/false-negative results were obtained, for example, if a result was inconsistent or unavailable (no spectrum), were also regarded as partially correct (no confirmation, but also no all clear). To calculate the overall accuracy index of the entire identification approach, a point system was introduced, giving one point for each correct identification result (green). Furthermore, cells with partially correct results (yellow) received half points, while no points were given for incorrect results (red). All points were then summed over the entire table; the sums were subsequently divided by the number of cells of each table. The quotient thus determined was finally multiplied by 100 and expressed as a percentage. To exclude an undue weighting of the measured data from laboratory XI, the point values from the rows “Laboratory XI” and “Laboratory XII” were averaged before summation in identification approach A.
The overall identification accuracy of identification approach A was 77% (see Table S2 in the supplemental material). While the accuracy of identification of samples 1 (B. pseudomallei), 4 (B. anthracis), 5 (Ochrobactrum anthropi), 7 (B. mallei), 8 (B. thailandensis), and 9 (Y. pestis) was relatively high, there were major problems when diagnosing samples 2 (F. tularensis subsp. holarctica), 3 (Brucella canis), 6 (Yersinia pseudotuberculosis), and 10 (B. thuringiensis). Furthermore, the results from laboratory IX were generally difficult to assess. In this laboratory, diagnoses were made only on the basis of the standard BioTyper database for clinical microorganisms; neither an in-house database of HPB nor the SR library from Bruker were available to this partner (cf. Tables S1 and S2).
The overall identification results improved significantly when spectra of the interlaboratory exercise were tested against the database of highly pathogenic microorganisms compiled at RKI over the past 10 years; the overall identification accuracy improved from 77.0% in approach A to 93.5% in approach B (see Table S3 in the supplemental material). The improvements were particularly striking in the cases of samples 2 (F. tularensis subsp. holarctica), 3 (B. canis), and 10 (B. thuringiensis). However, differentiation between samples 6 (Y. pseudotuberculosis) and 9 (Y. pestis) improved only slightly in approach B.
In the third approach, identification approach C, the overall picture did not differ much from approach B (see Table S4 in the supplemental material). The minor improvement in the overall identification accuracy (93.7%, compared to 93.5%) is statistically insignificant, which is not particularly surprising: although both approaches involved different software implementations with different algorithms, they relied on an identical spectral database. The results given in Table S4 demonstrate a decreased identification rate for sample 7 (B. mallei) and a slight improvement for sample 6 (Y. pseudotuberculosis). However, the major advantage of approach C over approach B consists in the fact that it allows analysis of spectra obtained by means of the bioMérieux/Shimadzu system (cf. rows “Laboratory XI” of Tables S3 and S4). Due to missing import capabilities for Shimadzu-specific spectral files in the BioTyper software, the data acquired by laboratory XI may be analyzed by approach C but not using the BioTyper software employed in identification approach B.
Figure 4 shows a summary of the results of all identification approaches. This table again illustrates the improvements of the identification accuracies in approaches B and C in comparison to those of approach A, particularly for samples 1 to 4 and 10. With regard to samples 2 (F. tularensis subsp. holarctica) and 3 (B. canis), we assume that the relatively high error rates in approach A derive from incomplete or missing spectral entries for both subspecies/species in the SR BioTyper library extension. We have noted that identification of F. tularensis subsp. holarctica and of B. canis was incomplete in cases where identification was made by means of this particular database extension. A closer examination of the SR database content revealed the absence of subspecies information in entries for F. tularensis (sample 2) and the lack of spectral entries for Brucella species other than B. melitensis (sample 3).
Summary of the different identification results from the MALDI-TOF MS ring trial, with the numbers of correct, partly correct, and incorrect identifications, as well as point scores for identification (correct identification, one point; partly correct, half a point; incorrect, zero points) and the corresponding identification accuracy values (percentages). Green, identification accuracy for the given microbial strain is equal to or greater than 90%; yellow, accuracy is equal to or greater than 75% and below 90%; red, accuracy is below 75%.
In contrast, it was interesting to note that the sophisticated software algorithms employed in approaches B and C can cause problems even in cases where extensive spectral databases are available. To give an example, differentiation between Y. pseudotuberculosis and Y. pestis by approaches B and C is far from being ideal (cf. samples 6 and 9 in Tables S3 and S4 in the supplemental material). To a certain extent, this could be caused by the low initial concentration of Y. pestis in the sample solution (1.3 × 107 CFU/ml) (Table 1). Several ring trial participants have indeed reported a relatively poor signal-to-noise ratio in MALDI-TOF mass spectra acquired from aliquots of sample 9. Low spectral quality is certainly a factor which makes differentiation of Y. pestis and Y. pseudotuberculosis difficult. An even more important factor, however, is the very high degree of similarity of spectra from these two very closely related species. In fact, Y. pestis is known as a clone of Y. pseudotuberculosis which has only recently evolved from Y. pseudotuberculosis (51, 52). Both species share genomic sequences and have identical 16S rRNA genes (53). As a consequence, their differentiation by MALDI-TOF MS is challenging; it has been found that differentiation can be carried out only on the basis of a single mass peak at m/z 3,065 (36, 38). This peak has been assigned to a fragment of the plasmid-encoded (pPCP1) Pla protein. Therefore, MS-based differentiation is possible only for strains of Y. pestis carrying the pPCP1 plasmid. At this point, it should be stressed that visual inspection of the mass spectra would have helped in solving the particular problem of differentiating Y. pseudotuberculosis and Y. pestis. Although the biomarker for Y. pestis at m/z 3,065 is typically very intense, pattern recognition routines do not always provide reliable results in cases when the outcome of the identification is based on the presence or absence of only a single biomarker. Following this line of reasoning, the supervised modeling approach chosen by laboratory XI, which relies on 15 biomarkers to discriminate between Y. pestis, Y. pseudotuberculosis, and Y. enterocolitica, may provide the basis for a more robust typing scheme (54).
In the present study, problems also occurred when differentiating the closely related members of the B. cereus group, B. anthracis, B. cereus, and B. thuringiensis. First, we have no information on whether MALDI-TOF MS allows reliable differentiation of B. cereus and B. thuringiensis. Our own observations, however, revealed that strains from both species are frequently identified based on their strain-specific spectral profiles. On the other hand, mass spectra of B. anthracis strains exhibit a specific β-SASP signal at m/z 6,679 (22, 24, 55–58) that is usually not present in spectra of other B. cereus group members. However, in the recent literature, there is increasing evidence that spectra of certain strains of B. cereus and B. thuringiensis may also exhibit β-SASP peaks at m/z 6,679 (59) (see also the spectrum of B. cereus BW-B in Fig. 1). Therefore, this β-SASP biomarker is not necessarily pathognomonic for B. anthracis. Furthermore, we and others have noted that the second published biomarker of B. anthracis, at m/z 5,413 (24), is often found in spectra of B. cereus and B. thuringiensis also. Both facts should be considered when assessing the identification results for B. cereus group members: the results of MALDI-TOF MS should not form the sole basis for potentially far-reaching decisions; for example, in the event of suspected intentional release of B. anthracis.
Conclusions.This paper reports on an interlaboratory external quality assurance exercise (EQAE) conducted by 11 partner institutions from nine European countries. In this ring trial, MALDI-TOF MS was used as a tool for rapid, reliable, and cost-effective identification of highly pathogenic microorganisms. In the preparatory phase of the exercise, pilot tests on nonpathogenic strains were carried out in order to optimize and standardize the experimental procedures at the partner institutions and to identify possible sources of underperformance. Irradiation by gamma rays proved to be a MALDI-TOF MS-compatible inactivation method which induced only subtle spectral changes with negligible influence on the quality of the diagnosis. In the ring trial, the average identification accuracy was 77% when using nonstandard mass spectral databases. The accuracy improved to >93% when spectral diagnoses were reached on the basis of an optimized spectral database with better coverage of highly pathogenic and related species.
The present EQAE has highlighted current strengths and weaknesses of the MALDI-TOF MS-based approach for identification of HPB and has confirmed the need for high-quality spectral databases to facilitate improved identification accuracy. Experiences gathered from the present international EQAE also suggest that, as long as high-quality and comprehensive spectral databases are available, different preparative procedures, the degree of user experience, and different types of instrumentation and analysis software are not likely to critically affect identification of HPB. The compilation of complete and comprehensive databases is thus considered to be of paramount importance for reaching accurate and reliable spectral diagnoses. Future efforts to improve the diagnostic capabilities should therefore focus on the exchange of validated reference spectra. We are confident that further ring trials will confirm the improvements achieved by such activities.
ACKNOWLEDGMENTS
We thank T. M. Fuchs (ZIEL, Technical University Munich, Germany), J. Rau (CVUA, Stuttgart, Germany), W. Beyer (University of Hohenheim, Stuttgart, Germany), A. Paauw (TNO, Rijswijk, Netherlands), M. Dybwad (NDRE, Kjeller, Norway), and N. Schürch (Labor Spiez, BABS, Spiez, Switzerland) for providing strains, samples, or spectra of important microbial pathogens. S. Weil, S. Zange, and B. Pályi are grateful to P. Hufnagl (AGES, Vienna, Austria), B. Thoma (InstMikroBioBw, Munich, Germany), and M. Iván (Semmelweis University, Budapest, Hungary), respectively. In addition, we thank S. Becker, P. Lochau, A. Schneider, S. Howaldt, and R. Andrich (all RKI, Berlin, Germany) for excellent technical assistance.
We are very grateful to the European Commission and CHAFEA for financially and technically supporting the QUANDHIP Joint Action (CHAFEA grant agreement number 2010 21 02). Parts of this work were supported by the Federal Ministry of Education and Research, BMBF (Förderkennzeichen/grant number 13N11166).
FOOTNOTES
- Received 27 March 2015.
- Returned for modification 21 April 2015.
- Accepted 1 June 2015.
- Accepted manuscript posted online 10 June 2015.
Supplemental material for this article may be found at http://dx.doi.org/10.1128/JCM.00813-15.
- Copyright © 2015, American Society for Microbiology. All Rights Reserved.
