Previous Article | Next Article ![]()
Journal of Clinical Microbiology, September 2008, p. 3056-3062, Vol. 46, No. 9
0095-1137/08/$08.00+0 doi:10.1128/JCM.00878-08
Copyright © 2008, American Society for Microbiology. All Rights Reserved.

Provincial Laboratory for Public Health (Microbiology), Calgary, Alberta, Canada,1 Department of Microbiology and Infectious Diseases, University of Calgary, Calgary, Alberta, Canada2
Received 7 May 2008/ Returned for modification 19 June 2008/ Accepted 9 July 2008
|
|
|---|
|
|
|---|
The development of kits for the detection of a panel of respiratory viruses using suspension microarrays has been described (1, 2, 9, 11-13, 16). These assays show good (and in some cases enhanced) performance compared with traditional antigen and culture methods for respiratory virus detection. Based on these studies, it was thought that these assays may be a suitable replacement for in-house NATs with good sensitivity, specificity, and savings in cost and time. Array approaches also have the added benefit of broad viral detection compared with our in-house NAT panel. The Luminex xTAG respiratory viral panel (RVP) used in this evaluation allows detection of a panel of 20 respiratory virus targets with differentiation between 19 of these (enteroviruses and rhinoviruses are detected by the generic picornavirus probe) (14). Here we report the results of a prospective and retrospective evaluation of the RVP assay compared with a panel of in-house real-time NATs.
|
|
|---|
The patient age range for study samples was evenly distributed from <1 year of age to >100 years. The most highly represented group was infants/children of ages 1 to <10 years, at 21.8% of specimens, as shown in Table 1. A breakdown of specimen type by age is included in Table 1.
|
View this table: [in a new window] |
TABLE 1. Specimen types included in the studya
|
In-house testing algorithm and NAT panel. According to the diagnostic algorithm for testing respiratory specimens at the ProvLab, NP samples were prescreened by DFA for influenza virus A (IFVA), IFVB, parainfluenza virus types 1 to 3 (PIV1-3), and respiratory syncytial virus (RSV) using monoclonal antibodies from Imagen (Lenexa, KS). The DFA-positive samples were not tested by NATs and were thus not included in this RVP comparison study. All other respiratory sample types (mainly TS and BAL) and DFA-negative NP samples were first tested by the in-house NAT panel and then by the RVP assay for comparison. Individual real-time NATs for comparison with RVP were directed against IFVA, IFVB, PIV1-4, RSV, human metapneumovirus (hMPV), and respiratory adenovirus (ADV) types. These methods have been published and are well evaluated in terms of assay sensitivity and specificity (4, 7, 10, 15, 18-21). Additional hemagglutinin subtyping by real-time reverse transcription-PCR (RT-PCR) using hydrolysis probes was undertaken for IFVA-positive samples by NAT (procedure provided by Yan Li, National Microbiology Laboratory, Winnipeg, Canada) as previously published (17) or as part of the RVP. If positive results for coronaviruses or picornaviruses were obtained in the RVP assay, the samples were screened subsequently by additional real-time NATs against these targets (10; ProvLab, unpublished data), since such testing is not part of the routine diagnostic panel.
RVP testing algorithm. Specimens tested by the RVP assay were DFA-negative NP samples and all other respiratory sample types that had been extracted and tested by in-house NATs (n = 1,530, with 865 tested prospectively and 665 tested retrospectively). RVP testing was undertaken according to the manufacturer's instructions (12, 14). The assay allows for detection of the following: IFVA, with additional subtyping of positives into H1, H3, and H5; IFVB; respiratory syncytial virus A and B (RSVA and RSVB); coronaviruses (severe acute respiratory syndrome [SARS] virus, NL63, 229E, OC43, and HKU1); PIV1-4; hMPV; picornaviruses (including enteroviruses and rhinoviruses), and a range of ADV types. In addition, the RNA bacteriophage MS2 can be used as an internal extraction/inhibition control, and the DNA bacteriophage lambda can be used as an amplification and assay performance control. The study specimens were extracted primarily for diagnostic testing; hence MS2 was not added prior to extraction. MS2 was spiked into the RT-PCR master mix as an RNA control for the amplification and detection steps. If the MS2 or lambda controls failed in the RVP assay, the test was repeated to obtain a valid result.
Discordant analysis. If results were discordant between in-house NAT and RVP, both tests were repeated concurrently to evaluate any problems relating to sample degradation or sporadic weakly positive results. Assignment of such samples as having concordant or discordant results was based on duplicate testing by both methods.
Statistical analysis. The SPSS software, v14.0 (SPSS Inc., Chicago, IL), was used for statistical analysis of data. The significance values were analyzed using Pearson chi-square analysis. The kappa value was computed to assess the agreement between the two testing methods.
|
|
|---|
Of the 690 positives by RVP, 638, 46, and 6 specimens were positive for a single virus, 2 viruses, and 3 viruses, respectively. Of the 643 positives by in-house NATs, 586, 54, and 3 specimens were positive for a single virus, 2 viruses, and 3 viruses, respectively. Of the 690 specimens that were positive by RVP, the in-house assays did not detect 88 picornavirus positives and 3 IFVA positives. Of the 643 specimens that gave a positive result by the in-house NATs, 48 were missed by RVP. The following targets were missed by RVP for these discordant specimens: 23 for ADV, 12 for RSV, 5 for IFVA, 3 for PIV, 2 for IFVB, 2 for ADV/RSV coinfection, and 1 for ADV/PIV coinfection.
The samples tested were from patients ranging in age from less than a year old to over 100 years; positive results were obtained for all the age groups tested. The majority of positive samples were from the age range 1 to <10 years (n = 246/720 [total positive by in-house NATs and/or RVP], 34.2%). The percentage of study samples tested in each age category together with positives identified by RVP and in-house NATs are given in Fig. 1.
![]() View larger version (19K): [in a new window] |
FIG. 1. Age distribution for samples tested and positives detected by in-house NATs and RVP. The numbers of positives detected in each age group by in-house NATs and RVP are shown. The line graphs indicate the proportion of study specimens tested by age group and the percentage positive by age category for the two test methods.
|
![]() View larger version (22K): [in a new window] |
FIG. 2. Sample type distribution for samples tested and positives detected by in-house NATs and RVP. The numbers of positives detected for each of the specimen types by in-house NATs and RVP are shown. The line graphs indicate proportions of study specimens tested by type and percentage positive by specimen type for the two test methods. NP, nasopharyngeal swab or aspirate; BAL, bronchoalveolar lavage; TS, throat swab; NS, nasal swab; ETT, endotracheal tube.
|
A total of 122 specimens (7.1%) failed to give a valid result for MS2 on initial testing. Chi-square analysis of the failure of MS2 in specimens that were positive for a respiratory viral target compared to specimens that were negative shows no significant difference in the failure rate (P = 0.0296). The lambda external positive control was used on 66 runs with 6 (9.0%) failures.
Analysis of individual targets. Results for targets tested routinely by in-house NATs and cross-tested by RVP are provided in Table 2. Since neither testing method is a true gold standard for detection of respiratory viral targets, we calculated the sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) for the in-house NAT using the RVP as a gold standard and vice versa.
|
View this table: [in a new window] |
TABLE 2. Assay concordance for individual targetsa
|
98.0%, as shown in Table 2. Kappa correlation statistics are also given in Table 2. The 95% confidence interval for detection of IFV by NAT compared to that by the RVP assay was 0.967 to 0.994. The in-house NAT used for detection of IFVA was performed by nucleic acid sequence-based amplification (NASBA) (10, 15), and typing was undertaken using a real-time hydrolysis probe-based assay. Of the five specimens IFVA positive by the NASBA assay that were missed by RVP, two specimens could not be detected by the real-time typing assay and the other three specimens had threshold cycle (CT) values of 30.3, 39.7, and 40.1. Three specimens with an IFVA-positive result by the RVP assay gave negative results by in-house NATs. Two of these specimens could not be subtyped by the RVP assay, and all three samples had a low median fluorescence intensity, indicating a low concentration of IFVA. Thus, discordant IFVA results were few and were likely due to a low viral load.
There were a total of 37 IFVB-positive samples in the study (37/1,530 or 2.4% positive). In order to have sufficient positive samples for analysis, 33 of the 37 positive samples were selected from stored specimens as part of the retrospective arm of the study. Testing for IFVB was performed using an in-house NASBA assay (10). Of the IFVB-positive samples, four (10.8%) were in addition positive for another respiratory virus, two with IFVA and two with a picornavirus. The positivity rate for IFVB was 2.4% for the in-house NAT and 2.2% for the RVP assay (37 and 35 positive samples, respectively), as shown in Table 2. Concordance between the in-house NAT and the RVP assay was 99.9% for IFVB, and the kappa correlation was 0.972 (Table 2). Sensitivity, specificity, PPV, and NPV were all
94.4%, as shown in Table 2. Both the IFVB-positive samples that were missed by RVP had a weak NASBA signal, and one specimen was coinfected with IFVA. Thus, discordant IFVB results were few and were likely due to a low viral load.
There were a total of 97 PIV-positive samples in the study (97/1,530 or 6.3% positive), as shown in Table 2. For 22 (22.7%) of the PIV-positive samples, another coinfecting virus was detected. The positivity rates were 6.3% for the in-house NAT and 6.1% for the RVP (97 and 93 positive samples, respectively). Concordance between the in-house NAT and the RVP was 99.7% for PIV, with good sensitivity, specificity, PPV, NPV, and kappa values, as shown in Table 2 (all
95.9%). The RVP assay identified 93 PIV-positive samples. The prevalences of the different PIV types as determined by RVP were as follows: PIV1 (n = 4), 4.3%; PIV2 (n = 15), 16.1%; PIV3 (n = 66), 71.0%; and PIV4 (n = 3) 3.2%. Both PIV1 and PIV3 were detected in five samples (5.4%).
Four specimens which were PIV positive by NASBA (7, 10) were not detected by the RVP assay. These were tested by an alternate real-time PCR (20; ProvLab, unpublished), with only two of the four giving positive results, both of which were weak (CT of 42.6 and 37.6). Two of the four specimens missed by the RVP assay had coinfections, one with ADV and one with RSV. Thus, discordant PIV results were few and were likely due to a low viral load.
There were a total of 92 RSV-positive samples in the study (92/1,530 or 6.0% positive). For 25 (27.2%) of the RSV-positive samples, another target virus was detected. The positivity rate was 6.0% for the in-house NAT and 5.1% for the RVP assay (92 and 78 positive samples, respectively). Of the 78 RVP-positive specimens, RSVA was detected in 63 (80.8%) and RSVB in 15 (19.2%). Concordance between the in-house NAT and the RVP was slightly less for RSV than for IFVA, IFVB, and PIV at 99.1%. Analyses (including sensitivity, specificity, PPV, NPV, and kappa values of each assay) are given in Table 2.
Of the 14 RSV positives that were missed by RVP, 5 were present as coinfections with other viruses. The CT range for RSV using an in-house PCR (19) in the samples missed by RVP ranged from 34.0 to 43.0, with a mean of 37.7 and standard deviation of 2.9, indicating that the specimens with a lower copy number were missed. The age range for these discordant specimens was from less than 1 year to 54 years, with representatives from every age group. The specimen types for which the positives were missed included five DFA-negative NP samples, seven TS samples, and two BAL samples, suggesting that there was no bias in the specimen type for detection of RSV positives.
There were a total of 47 hMPV-positive samples in the study (47/1,530 or 3.1% positive). For 14 (29.8%) of the hMPV-positive samples, another target virus was detected. The positivity rate was 3.1% for both the in-house NAT and the RVP assay (47 positive samples by both methods). Concordance between the in-house NAT and the RVP assay was 100%, as shown in Table 2.
There were a total of 62 ADV-positive samples in the study (62/1,530 or 4.1% positive). For 25 (40.3%) of the ADV-positive samples, another target virus was detected. The positivity rate was 4.1% for the in-house NAT and 2.4% for the RVP assay (62 and 36 positive samples, respectively). Concordance between the in-house NAT and the RVP assay was 98.3%, with lower sensitivity, specificity, PPV, NPV, and kappa values than with the other respiratory virus targets (Table 2).
The 36 samples that tested positive for ADV by RVP showed a variation in viral load, with a mean CT value of 28.0, standard deviation of 6.7, and range from 17.5 to 42.6, using our in-house PCR (21). A total of 26 ADV NAT-positive specimens gave negative results by RVP, with 15 of these specimens having more than 1 viral target present. Of the missed samples, one had a CT value of 18.8, and the CT range for the remainder of the samples was from 30.9 to 41.3. The mean CT value of samples missed by RVP was 34.8, and the standard deviation was 5.0. Five discordant samples were sequenced for the partial hexon gene (21); all sequences belonged to species C, with four of serotype 1 (including the sample with a CT value of 18.8) and one of serotype 2. The detection of ADV in patients between the age range of 1 to <10 years was significantly lower by the RVP assay than was the case with the other age groups (P < 0.005). The most common specimen type from young patients is NP swabs; thus, we analyzed the difference in the rate of ADV detection in NP and non-NP specimens; however, no significant difference was noted (P = 0.09).
The 1,530 samples included in this study were tested for picornaviruses by RVP. Tests for this virus family are not routinely included in our diagnostic testing algorithm. A total of 184 (12.0%) samples were positive for picornaviruses by RVP; 35 (28.3%) of these samples were also positive for other targets. The positive samples were tested retrospectively by in-house NASBA (6, 10), with 96/184 (52.2%) of the picornavirus positives confirmed in this way. The in-house NATs thus detected only about half the picornavirus positives identified by the RVP assay, of which 21 specimens were coinfections. Based on the in-house NAT, 95.8% of the picornaviruses detected were rhinoviruses and 4.2% were enteroviruses. Samples picornavirus positive by the RVP assay were subjected to sequencing of the 5' noncoding and VP1 regions, and >85.0% were confirmed as rhinoviruses (data not shown). Thus, the discrepancy between RVP and in-house NATs for picornaviruses was largely due to poor identification of rhinoviruses by the in-house NASBA.
Coronavirus testing is not included in the current in-house NAT panel test. The 1,530 study samples tested by the RVP assay yielded 26 samples positive for a coronavirus (1.7% of all samples). The breakdown of positives for the different types of coronaviruses was as follows: for 229E, n = 11; for OC43, n = 8; for HKU1, n = 4; and for NL63, n = 3. Only these RVP-positive samples were tested by in-house NATs for coronaviruses (10; ProvLab, unpublished) with concordant results. For 5 (19.2%) of the 26 coronavirus-positive samples, another virus was detected by RVP and/or in-house NAT.
A total of 70 specimens were positive for more than one respiratory virus, as identified by RVP or in-house NATs. Of these, 61 specimens were positive for 2 targets and 9 were positive for more than 2 targets. Both the in-house NATs and RVP identified coinfections in 39 (52.9%) specimens. Additionally, the RVP identified 11 coinfected samples, giving the RVP a total coinfection detection rate of 71.4% (50/70 coinfections). The in-house NATs identified a total of 55 coinfections, giving the NATs a coinfection detection rate of 78.6% (55/70 coinfections). The in-house NATs missed identification of coinfections for 11 specimens, all of which were positive for picornaviruses by RVP. The RVP assay missed 16 of the coinfected samples identified by in-house NATs; 10 of these were positive for ADV, 2 for RSV, and 1 each for IFVB and PIV, and 2 were ADV/RSV coinfections. Four specimens had coinfections, but neither of the assays identified all the coinfecting viruses. The in-house NATs missed the presence of picornaviruses in all four samples, and the RVP did not detect the presence of ADV in two samples, RSV in one sample, and ADV/PIV in one sample.
Specificity check for SARS and IFVA H5 testing. All 1,530 specimens included in the study were screened for IFVA subtype H5 and the SARS coronavirus by RVP. There were no false-positive results in this screen.
Hemagglutinin typing of IFVA-positive samples. In-house NATs identified 246 IFVA-positive specimens, with RVP identifying 244 for a total of 249 by either method. Of the 246 specimens positive by NASBA, 241 (98.0%) could be successfully typed as subtype H1 (n = 147; 61.0%) or subtype H3 (n = 94; 39.0%) using in-house real-time NATs. Of the 244 positive by RVP, 212 (86.9%) were typed concurrently in the assay as subtype H1 (n = 140; 66.0%) or subtype H3 (n = 72; 34.0%). The samples that were discordant for typing between the two methods were low-level positives with a late crossing point (CT range from 33.3 to 43.2 with a mean of 37.9 and standard deviation of 2.9) by NAT. There was no evidence that IFVA-positive samples missed by RVP were skewed toward a particular subtype, but discrepant results were more likely due to a low viral load. Combining both methods, 239/241 (99.2%) IFVA-positive specimens could be successfully typed as subtype H1 (60.6%; n = 146) or subtype H3 (38.6%; n = 93) and 2 samples (0.8%) were not subtyped by either method and would require amplification by culture before further analysis.
|
|
|---|
A comparison for the detection of IFVA, IFVB, hMPV, and PIV shows that the RVP assay meets our current needs for diagnostic sensitivity and specificity for these targets. All the positive specimens for these targets that were missed by RVP were weak positives by in-house NATs. A previous study comparing the RVP assay to DFA/culture for detection of respiratory viruses reported an increase of 43% in sensitivity (12).
Target detection in the different sample types was similar by the two methods, with a higher proportion of the NP specimens being positive than was the case for the BAL specimens. The reason for higher positivity in NP samples could be that they are collected from younger children with respiratory symptoms; BAL specimens may be collected from patients with disease processes other than viral infections. In addition, the use of saline for BAL collection could dilute any virus in the sample.
The sensitivity of the RVP assay for the detection of RSV was less than that for other targets when in-house NAT was used as the gold standard. By excluding RSV DFA-positive NP samples from our analysis, we challenged the RVP assay sensitivity much more rigorously than in previous studies, and this was reflected in the fact that all the discordant samples were weak positives with high CT values by RT-PCR. Positives were missed in all the age groups and specimen types tested, suggesting that there was no bias for detection of RSV. Mahony et al. (12) have reported a sensitivity of 96.6% (28/29) for the detection of RSV compared to the traditional DFA/culture methods.
Our study shows a drop in sensitivity for the detection of ADV compared with in-house PCR. As shown in the results, there was a significant reduction in the detection of ADV in younger patients by the RVP assay; however, there was no significant difference based on sample type, and our analysis of serotypes was too limited to draw complete conclusions. The in-house PCR-positive samples missed by the RVP assay were mainly those with a low viral load, but a skew toward reduced pickup of ADV species C has been reported (RVP package insert; data not shown), and this may account for the one sample with a low CT value that was missed by the RVP assay. We will continue to monitor young patients with suspected ADV infections by in-house NATs. Sensitivity for ADV detection compared with DFA/culture has been reported previously as 100% (12). Such discrepancies can be accounted for by different patient populations and geographical and temporal differences in ADV serotype distribution. Further work is being undertaken to adjust the ADV component of the assay to ensure sensitive pickup of all serotypes responsible for respiratory infection. The RVP assay can detect the presence of SARS coronavirus and IFVA subtype H5. No false positives were obtained for these targets, showing the RVP assay to be specific. The clinical specificity of the RVP assay for detection of IFVA subtype H5 has been shown to be 100% in other studies as well (9).
MS2 can be used as an extraction control for the RVP assay, but in this study we were only able to spike the MS2 RNA into the PCR master mix; therefore, we cannot comment in detail on the failure rate for MS2 based on our experiments. Failure to detect MS2 in our experiments is likely the result of the freeze-thaw of MS2 RNA, and prospective testing using MS2 prior to extraction has given a much lower failure rate than that seen for this study (data not shown). There was no significant difference in the failure rate for MS2 between specimens positive or negative for other viral targets, suggesting that competitive inhibition of MS2 was not a problem. Failure of the lambda external control was seen more at the beginning of the study, and storage of lambda DNA in single-use aliquots greatly improved its performance. No PCR contamination issues were encountered over the 6-month study period.
By excluding DFA-positive NP samples from our study, we have enhanced the available diagnostic data for the RVP assay and have confirmed its excellent performance for routine testing of IFVA, IFVB, PIV1-3, and RSV. A detailed workflow and economic analysis are under way as a second part to this study. Preliminary studies show that use of the RVP assay will result in cost savings and the workflow is much better than that involved in undertaking individual or small multiplex in-house NATs to cover the broad range of respiratory viral targets. Results for 20 viral targets are available within 8 h of receiving specimens. The RVP assay is comprehensive and includes all the respiratory viral targets that are currently tested routinely for the diagnosis of acute respiratory tract infections. The technology is flexible and can easily allow for incorporation of other targets (e.g., human bocavirus) in the future. The assay is sensitive and specific and has high throughput. It allows for the use of 96-well plates (allowing batching as appropriate) and can be used for detecting coinfections.
Results of this study indicate that implementation of the RVP for testing of respiratory samples will improve our rate of diagnosis for respiratory tract infections. The Luminex xTAG RVP assay has recently been approved by the FDA for the most common viral targets (IFVA with H1/H3 subtyping, IFVB, PIV1-3, RSV, rhinoviruses, hMPV, and ADV). Such approval is useful for diagnostic laboratories and ensures ongoing quality. Based on the results of this and previous studies, we have implemented the use of RVP as part of our diagnostic testing algorithm from February 2008 with the caveat that we intend to perform additional testing for those at high risk for systemic ADV infection (especially immunocompromised patients) until a further adaptation of the RVP assay addresses the slight loss in sensitivity for some serotypes.
We thank the technologists at the Provincial Laboratory for Public Health (Molecular Diagnostics) for their assistance in performing the in-house nucleic acid tests.
Published ahead of print on 16 July 2008. ![]()
|
|
|---|
| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Copyright © 2009 by the American Society for Microbiology. For an alternate route to Journals.ASM.org, visit: http://intl-journals.asm.org | More Info»