ABSTRACT
Herpes simplex virus (HSV) causes acute and relapsing symptoms characterized by ulcerative lesions. Laboratory diagnosis of HSV in cutaneous or mucocutaneous lesions has historically been performed with the use of viral cell culture systems; however, these tests are laborious and suffer decreased sensitivity for advanced-stage lesions. The recent availability of FDA-cleared moderately complex assays has resulted in the increased use of molecular diagnostics for the routine detection of HSV in superficial swab specimens. We performed a clinical evaluation of the recently FDA-cleared illumigene HSV 1&2 loop-mediated isothermal amplification (LAMP) assay (Meridian Bioscience, Cincinnati OH) for the detection and differentiation of HSV-1 and HSV-2 in cutaneous and mucocutaneous swab specimens. A total of 1,153 clinical swab specimens were collected and tested at 7 different clinical centers. Each specimen was tested for the presence of HSV-1 and HSV-2 using the illumigene assay, and results were compared to those of the enzyme-linked virus-inducible system (ELVIS) as the reference method. Overall, the illumigene assay demonstrated a sensitivity and specificity of 94.8% and 95.5%, respectively, for the detection of HSV-1. Detection of HSV-2 was similar, with a sensitivity of 98.9% and a specificity of 95.5%. Discrepant analysis was performed using an alternative molecular test (AmpliVue HSV1+2 assay; Quidel Molecular, San Diego, CA) on 91/99 specimens that were recorded as false positive (FP) or false negative (FN) compared to the reference method. In total, 57/78 (73%) FP and 9/13 (69%) FN illumigene results were supported by the AmpliVue result. The illumigene HSV 1&2 assay demonstrated high sensitivity and specificity to detect and differentiate HSV in clinical specimens and identified 57 additional specimens that were positive for HSV compared to culture. The use of LAMP eliminates the need for the cycling of temperatures and provides results in less than 60 min, with approximately 2 min of hands-on time per specimen.
INTRODUCTION
Herpes simplex virus 1 (HSV-1) and 2 (HSV-2) are double-stranded DNA viruses of the Herpesviridae family and are a common cause of cutaneous and mucocutaneous oral and genital lesions (1, 2). Historically, HSV-1 has been associated with oral lesions while HSV-2 has more often been associated with genital lesions, although recently this epidemiology has been changing. Additionally, HSV can cause invasive disease, including sepsis and fulminate encephalitis in neonates that are infected by exposure to herpetic lesions present at the time of birthing (3). Infection with HSV-1 is common, with seroprevalence reaching 50% to 70% in developed countries and >80% in developing countries (4–6). The seroprevalence of HSV-2 varies from 5% to 40% (7, 8). While there is no cure for HSV infections, specific antiviral therapy exists to alleviate symptoms, shorten the duration of recurring outbreaks, and to treat life-threatening manifestations (9). Therefore, accurate laboratory diagnosis of these infections is important.
The diagnosis of HSV based solely on clinical exam is difficult. Typical HSV lesions can mimic those of other sexually transmitted infectious agents (e.g., Treponema pallidum and Haemophilus ducreyi), staphylococcal folliculitis, herpes zoster, and skin abrasion due to physical injury (10). Therefore, it is essential to confirm all possible herpetic infections with laboratory diagnostics. Classically, laboratory diagnosis of superficial HSV infections relied on the collection of vesicular exudate or scrapings from a lesion and inoculation into a susceptible cell line (Hep-2, A549, etc.). Following incubation, cells are observed microscopically for viral-induced cytopathic effects (CPE) over 5 to 14 days (11). Standard cell culture was improved with the development of the enzyme-linked virus-inducible system (ELVIS; Diagnostic Hybrids, Athens, OH), which increased sensitivity and reduced turnaround time (TAT) to approximately 24 h (12, 13). Overall, the sensitivity and specificity of ELVIS has been reported to range from 88% to 100% and 98% to 100%, respectively (14, 15). Importantly, the sensitivity of all culture-based diagnostics is dependent upon the presence and load of viable virus in a specimen, which is affected by the age of the lesion and specimen handling (16–18).
Molecular assays include laboratory-developed and FDA-cleared multiplex assays for simultaneous detection of and differentiation of HSV-1 and HSV-2 in superficial specimens (19–21). These assays are more sensitive and reduce TAT compared to viral culture and ELVIS (22–24). We present the first clinical evaluation, to our knowledge, of the recently FDA-cleared illumigene HSV 1&2 assay (Meridian Bioscience, Cincinnati, OH). In this study, a total of 1,158 residual cutaneous or mucocutaneous swab specimens from 7 clinical sites were collected for HSV testing. All specimens were tested using the illumigene HSV 1&2 assay, and results were compared to ELVIS as the reference method. Overall performance was determined by calculating the sensitivity and specificity of the assay. A subset of specimens was also analyzed in parallel using a real-time PCR (RT-PCR)-based laboratory-developed test (LDT) as a molecular comparator.
MATERIALS AND METHODS
Collection of HSV specimens.Residual swab specimens collected in viral transport media (including M4, M4RT, M6, UniTranz-RT, UTM, universal viral transport [UVT], and viral transport media [VTM]) and submitted to the clinical laboratory for routine HSV testing were enrolled in the study. Swabs from lesions of cutaneous (arms/hands, breast, back, abdomen, penile) and mucocutaneous (anorectal, vaginal, cervical, nasal, ocular, urethral, and oral) sources were acceptable specimens; no cerebral spinal fluid specimens were enrolled. All illumigene testing was performed on fresh specimens at each enrollment site. Corresponding ELVIS testing was performed on fresh specimens when possible; however, logistical aspects of the study design resulted in 589 specimens (50.8%) being tested by ELVIS following a freeze cycle. Fresh specimens were defined as specimens stored at 2°C to 4°C for up to 72 h. Frozen specimens were stored at −80°C before testing and were only thawed once. Fresh and frozen testing modalities are acceptable according to manufacturer recommendations. Specimen collection and testing were conducted at 7 different geographical locations (Sacred Heart Hospital, Pensacola, FL; The Medical College of Wisconsin, Milwaukee, WI; Pathology Inc., Torrance, CA; Cleveland Clinic, Cleveland, OH; CompuNet Clinical Laboratories, Moraine, OH; LeBonheur Children's Hospital, Memphis, TN; and Nationwide Children's Hospital, Columbus, OH). All testing for this study was performed using deidentified residual specimens after standard of care testing was completed. Each site performed testing in accordance with site-specific institutional review board approved protocols.
Enzyme-linked virus-inducible system.ELVIS testing was performed using the ELVIS D3 typing kit (Diagnostic Hybrids, Athens, OH) in accordance with the package insert protocol using standard shell vials (Diagnostic Hybrids, Athens, OH). The presence of blue cells due to the ELVIS β-galactosidase reaction was used as an indication of HSV-positive specimens. Positive specimens were then typed using fluorescently tagged antibodies contained within the kit. Positive specimens that could not be typed were reported as untypeable. Sites C, E, F, and G were unable to perform ELVIS testing onsite, so specimens were shipped frozen to site A for reference testing.
illumigene HSV 1&2 assay.The illumigene HSV 1&2 assay is a molecular assay that utilizes loop-mediated isothermal DNA amplification (LAMP). The assay targets a 208-bp sequence of the HSV-1 glycoprotein G and a 189-bp sequence of the HSV-2 glycoprotein G gene. Two test devices are used for the assay: one containing lyophilized reagents for detection of HSV-1 and the second containing reagents for detection of HSV-2. Testing using illumigene was performed following manufacturer recommendations. Specimens are reported as positive for HSV-1 or HSV-2 target DNA, negative, or invalid. All invalid results were repeated once by repeating preparation from the patient specimen.
Discrepant resolution.Specimens that demonstrated discrepant results between illumigene HSV 1&2 and ELVIS were resolved using the FDA-cleared AmpliVue HSV1+2 assay (Quidel Molecular, San Diego, CA) as an alternative molecular test. All discrepant testing was performed blinded at a single reference site (Meridian Bioscience Inc.), and all tests were performed from frozen specimens.
Laboratory-developed real-time PCR assay.A laboratory-developed test based on HSV-1 and HSV-2 analyte-specific reagents (Cepheid, Sunnyvale, CA) and SmartMix HM beads (Cepheid, Sunnyvale, CA) that contained Taq polymerase, deoxynucleoside triphosphate (dNTPs), and MgCl2 (mastermix) was used as a molecular comparator for a subset of fresh clinical specimens (n = 363). Specimens in VTM were prepared by 10 min of heat inactivation at 95°C, and 5 μl of heat-treated specimen was combined with 15 μl of mastermix. Amplification and detection were carried out on a SmartCycler II (Cepheid, Sunnyvale, CA) with the following cycling conditions: 1 cycle heating at 95°C for 15 s and 50 cycles of 95, 57, and 72°C for 15 s each. Specimens were identified as positive if either fluorescence for HSV-1 or HSV-2 targets had a cycle threshold (CT) of ≤45 and both the positive and negative external controls passed.
Statistical analysis.Results from the illumigene HSV 1&2 assay were compared to those of ELVIS culture. Performance characteristics, including sensitivity and specificity, were calculated using standard methods. Ninety-five percent confidence intervals (CIs) were calculated by using a binomial expansion.
RESULTS
Specimen characteristics and prevalence.A total of 1,158 specimens that met study criteria were used to determine assay performance. Overall, the prevalences of HSV-1 and HSV-2 based on ELVIS results were 21.7% and 15.5%, respectively. Enrolled specimens were collected from pediatric (≤18 years of age, n = 208) and adult (>18 years of age, n = 947) patients. Patient age was unknown for three specimens. The prevalences of HSV-1 and HSV-2 varied between age groups, lesion types, and virus types, ranging from 0% to 43% (Tables 1 and 2). Two specimens were positive for both HSV-1 and HSV-2 by illumigene and were removed from statistical calculations due to an inability to detect coinfection using ELVIS. One specimen that was positive for HSV by ELVIS but could not be typed was removed from analysis. The final number of specimens used to establish the performance of illumigene HSV 1&2 was 1,153.
HSV prevalence in cutaneous specimens by patient age
HSV prevalence in mucocutaneous specimens by patient age
Comparison of illumigene to ELVIS for the identification of HSV-1 and HSV-2.The performance of the illumigene assay to detect HSV-2 was calculated by comparing the illumigene assay result to that of ELVIS, which was reported as the true value (Table 3). The illumigene reported a positive result for HSV-2 in 179/181 (98.9%) specimens classified as HSV-2 by ELVIS. Target nucleic acid was not detected in the remaining 2 specimens (i.e., false negative [FN]). The illumigene assay identified an additional 44 specimens as positive for HSV-2, which were classified as negative by ELVIS (i.e., false positive [FP]). These data resulted in an overall sensitivity of 98.9% (CI, 96% to 100%) and specificity of 95.5% (CI, 94% to 97%) for HSV-2.
Performance of illumigene HSV 1&2 assay compared to that of ELVIS culture
ELVIS typing uses the same fluorophore for typing HSV-1 and HSV-2; therefore, a specimen that is positive for HSV-2 is not tested for HSV-1. Given this limitation, all specimens typed as HSV-2 by ELVIS were removed from the data analysis used to calculate the performance of the illumigene HSV 1&2 assay for HSV-1 detection. Removal of the 181 specimens typed as HSV-2 by ELVIS decreased the total specimens included for HSV-1 to 974. The illumigene assay reported a positive result for HSV-1 in 200/211 (94.8%) specimens also typed as HSV-1 by ELVIS. The illumigene detected HSV-1 in an additional 34 specimens that were classified as negative by ELVIS. These data resulted in a sensitivity of 94.8% (95% CI, 91% to 97%) and a specificity of 95.5% (95% CI, 94% to 97%) for the detection of HSV-1 compared with ELVIS (Table 3).
The sensitivity and specificity of illumigene for HSV were statistically equivalent between the pediatric (<18 years of age) and adult groups (P > 0.05), with the exception of specificity for HSV-2. In adults, the specificity was determined to be 94.8% (740/781) while in pediatrics the specificity was 98.4% (188/191) (P = 0.045). Following discrepant resolution, no statistical difference in assay performance was observed between age groups. Therefore, the initial difference may be attributable to the reduced sensitivity of the ELVIS reference method in adults.
Culture-based prevalences of HSV-1 (7% to 28%) and HSV-2 (6% to 30%) were variable among the clinical trial sites. This may have impacted site-specific performance statistics; specifically, one of two sites with <90% sensitivity for HSV-1 (site C) enrolled only 90 specimens. The other site with <90% sensitivity for HSV-1 (site E) enrolled 254 specimens but was also the only site to demonstrate <100% sensitivity for detection of HSV-2. These data would suggest a potential site-specific difference in laboratory practice or specimen collection that negatively impacted the assay results.
Performance of illumigene HSV 1&2 for lesion source.The performances of molecular assays can be affected by inhibitory substances present in clinical specimens. Eight types of specimens were enrolled in this study that represented various anatomic locations and matrices, with the most common being genital, including cutaneous (penile) and mucocutaneous (vaginal/cervical), followed by superficial skin lesions and oral sources (Table 4).
Performance of illumigene HSV 1&2 assay based on site of lesion
The illumigene assay performed similarly independent of specimen source or location. Both genital cutaneous and mucocutaneous specimens demonstrated >99% sensitivity for HSV-2 detection. Sensitivity for the detection of HSV-1 was higher for mucocutaneous specimens (93.0%) than for cutaneous specimens (75.0%); however, this may be the result of a comparatively low cutaneous genital HSV-1 prevalence, with only 8/72 specimens positive for HSV-1. The sensitivity for HSV-1 was 100% in oral lesions. Detection from superficial skin lesions was also sensitive for HSV-1 (97.7%) and HSV-2 (100%).
Overall, the specificity from each source was consistent with the majority of sources having >90% specificity prior to discrepant resolution. Two sources, cutaneous genital (HSV-2) and urethral (HSV-1) were below 90% specificity. In each of these data sets, prevalence was low. Collections of specimens from anorectal, nasal, ocular, and urethral sources were included in the data, but total specimen enrollment from these sources was low (5 to 46 specimens), which may not rigorously test the assay's performance for these sites.
Discrepant analysis.Discrepant analysis was performed on all specimens with discordant results between illumigene and ELVIS using the AmpliVue HSV1+2 assay (Quidel, San Diego, CA). The AmpliVue results were in agreement with illumigene in 78% (25/32) of HSV-1 specimens and in 82.0% (32/39) of HSV-2 specimens that were initially classified as FP (Table 5). Additionally, AmpliVue results were in agreement with illumigene in 80% (8/10) of HSV-1 specimens and in 50% (1/2) of HSV-2 specimens that were initially classified as FN. These data suggest that the majority of FP results likely contained the HSV amplicon, which is consistent with the increased sensitivity of molecular assays compared to that of cell culture (22, 25).
Discrepant analysis using Quidel AmpliVue HSV1+2 assay as an alternative molecular test
Thresholds of detection for ELVIS and the illumigene assay for the detection of HSV.A subset of specimens was also tested using a laboratory-developed RT-PCR test, which is used as the standard of care at site B (see Materials and Methods). Cycle threshold (CT) values were collected for all fresh specimens, and the percent positive agreements (PPAs) for ELVIS and illumigene were calculated at three CT categories: ≤30, ≤35, and ≤45 (Table 6). Frozen specimens were not included in this analysis since a freeze-thaw cycle may reduce the sensitivity of ELVIS and prevent a fair comparison of the assays.
Percent positive agreement of ELVIS and illumigene compared to LDT based on RT-PCR cycle threshold value
Among the 254 fresh specimens tested, 119 were identified as positive for HSV by the LDT. Of the 119 specimens that tested positive by LDT, 67.2% (80/119) had CT scores of ≤30, indicating a relatively high viral burden. The ELVIS results were in agreement with the LDT results for 78/80 (97.5%) specimens in this group, and the illumigene assay agreed with the LDT for all 80 (100%). An additional 22 specimens were positive according to the LDT with a CT of 30 to 35, indicating a medium to low viral burden. Including these specimens in the analysis, ELVIS detected HSV in 91/102 (89.2%) specimens, while illumigene detected HSV in 99/102 (97.0%) specimens with CT values of ≤35. Finally, 17 specimens were positive by LDT with CT values of >35 to 45, indicating a very low viral burden. The illumigene assay detected HSV in 5/17 (29.4%) specimens, while ELVIS detected HSV in only 2/17 (11.8%). The two positive ELVIS cultures were also positive by illumigene. Including all of the LDT-positive HSV specimens, ELVIS detected 93/119 (78.2%) specimens whereas illumigene detected 104/119 (87.4%) specimens.
DISCUSSION
The data from our study are consistent with other publications that demonstrate the increased sensitivity of molecular detection methods for HSV compared to that of cell culture. This is illustrated by the large number of specimens testing positive by illumigene but negative by ELVIS. To support the presence of HSV nucleic acid in this cohort, 83 specimens with sufficient volume for additional analysis were tested using another isothermal amplification assay based on a different HSV genetic target (AmpliVue HSV1+2; Quidel). The AmpliVue result was in agreement with illumigene for 57 of 83 (78.1%) specimens, generating initial discrepant results between illumigene and ELVIS. The remaining eight specimens were unavailable for discrepant analysis (7 mucocutaneous and 1 cutaneous). A limitation of the discrepant analysis was that all specimens were frozen prior to AmpliVue testing. This may have reduced the sensitivity of AmpliVue since freeze-thaw cycles may reduce sensitivity of molecular assays. Despite this limitation, the overall specificity of illumigene post discrepant resolution rose to 98.8% for HSV-1 and HSV-2.
The impact of viral burden present in the clinical specimen on the sensitivity of culture was clearly demonstrated by the PPA when comparing ELVIS to the LDT or illumigene in relationship to CT values. ELVIS culture detected HSV in only 59.1% of specimens with high CT values (30 to 35) and was further diminished when CT values were ≥35. The loss of low-level detection by ELVIS may be clinically significant since viral burden can differ based on several factors, such as age of the lesion, state of infection (primary versus reactivation), virus (HSV-2 versus HSV-1), and quality of specimen collection (17, 26, 27). If ELVIS was used as the standard of care at site B, 26 patients would have been misdiagnosed, potentially leading to undertreatment and an increase in the spread of the virus. Importantly, the illumigene assay was positive in 11 of these ELVIS negative specimens. Without clinical and patient data, such as stage of lesion and history of antiviral therapy, it is difficult to assess the relevance of these very low-viral-load specimens. These may represent nonviable virus in resolving lesions, environmental (amplicon) contamination, or poorly collected specimens.
There were some limitations to this study. The FDA claim for the illumigene assay includes a wide variety of cutaneous and mucocutaneous sources, including skin, genital, anorectal, ocular, urethral, nasal, and oral lesions. During this evaluation, a limited number of nasal, ocular, and urethral specimens were received and tested. Culture-based prevalence of HSV for all three sources was 0%, preventing an accurate calculation of illumigene sensitivity in these specimens. Similarly, only a single oral specimen was positive for HSV-1 by ELVIS. In these instances, the calculated sensitivities, specificities, and 95% confidence intervals may not be a true reflection of illumigene performance.
In addition to the increased sensitivity offered by illumigene, the hands-on time and total time to result are also reduced. ELVIS and other cell culture methods require manual setup and microscopic examination, which can be subjective and requires a minimum of 24 h for final results. The illumigene assay provides a result within 1 h from the time the specimen is received by the laboratory, requires approximately 2 min of hands-on time per specimen, and is capable of simultaneous analysis of 5 patient specimens per illumipro-10 analyzer. This test may be a good fit for laboratories that want the flexibility of on-demand or small-batch testing with a “moderate complexity” molecular assay.
FOOTNOTES
- Received 4 March 2016.
- Returned for modification 2 April 2016.
- Accepted 13 May 2016.
- Accepted manuscript posted online 18 May 2016.
- Copyright © 2016, American Society for Microbiology. All Rights Reserved.
