Previous Article | Next Article ![]()
Journal of Clinical Microbiology, March 2007, p. 715-720, Vol. 45, No. 3
0095-1137/07/$08.00+0 doi:10.1128/JCM.01264-06
Copyright © 2007, American Society for Microbiology. All Rights Reserved.

Clinical Research Unit, Department of Infectious & Tropical Diseases, London School of Hygiene and Tropical Medicine, Keppel Street, London WC1E 7HT, United Kingdom,1 Laboratório de Virologia do Instituto de Medicina Tropical and Departamento de Moléstias Infecciosas e Parasitárias da Faculdade de Medicina, Universidade de São Paulo, Av. Dr. Eneas de Carvalho Aguiar 470, Sao Paulo, Brazil,2 The Cancer Research UK Viral Oncology Group, Wolfson Institute for Biomedical Research, University College of London Medical School, Cruciform Bldg., Gower Street, London WC1E 6BT, United Kingdom,3 Medical Statistics Unit, Department of Epidemiology and Population Health, London School of Hygiene and Tropical Medicine, Keppel Street, London WC1E 7HT, United Kingdom4
Received 20 June 2006/ Returned for modification 28 November 2006/ Accepted 14 December 2006
|
|
|---|
|
|
|---|
In the absence of a culture system derived from KS tumor-associated cell lines to allow direct demonstration of KSHV (2, 18), the diagnosis of KSHV infection has relied on molecular techniques and the use of serological assays to detect antibodies to KSHV antigens produced during latent or lytic phases of the viral life cycle (13, 15, 26). The immunofluorescence assay to detect anti-LANA antibodies (IFA-LANA) has long been considered a reference serological test to detect KSHV infection (23). However, with a sensitivity as low as 64 to 67% (9, 29) among AIDS patients with KS, this first-generation assay alone is of limited use in large seroepidemiological studies. KSHV latent antigen (LANA-1) (31) and lytic products associated with ORF 65 (35) and ORF K8.1 (6) are the most immunogenic antigens described so far and have thus been included in second-generation serological assays, such as enzyme-linked immunosorbent assays (ELISAs) and Western blotting (39). However, given the great variety of antibody responses to KSHV antigens and the lack of a clearly defined "gold standard" to compare new tests, information obtained from various serosurveys can be confusing. The assessment of the performance of newly developed assays is required in order to allow the conduct and interpretation of studies regarding the epidemiology and pathogenesis of KSHV. We therefore conducted a comparative evaluation of commercial kits and in-house KSHV serological assays of the first and second generations produced in specialized virology laboratories in the United Kingdom and Brazil.
|
|
|---|
Seven groups of patients were selected based on their posterior probability of being KSHV infected or uninfected, as determined by clinical data and IFA-LANA results (Table 1). Group 1 included 57 AIDS patients with histologically proven KS; group 2 included 60 AIDS patients from São Paulo who previously tested KSHV seropositive by in-house (IMT) IFA-LANA; group 3 included 60 AIDS patients from São Paulo with unknown KSHV serostatus but an absence of KS lesions; group 4 included 60 patients attending a São Paulo clinic for sexually transmitted infections; group 5 included 47 HIV-negative men who have sex with men enrolled in a cohort study for HIV prevention in São Paulo; group 6 included 60 blood donors from Salvador, Bahia State; and group 7 included 105 infants (1 to 2 years old) from São Paulo who had participated in a measles vaccine research program.
|
View this table: [in a new window] |
TABLE 1. KSHV seropositivities obtained with eight different serological assays, by population group
|
(b) UCL IFA-LANA. UCL IFA-LANA was performed using antigens obtained from the B-cell PEL cell line (BCP-1) latently infected with KSHV. A human herpesvirus 8 (HHV-8)- and Epstein-Barr virus-negative B-cell line (Ramos) was used as a control. Sera were diluted in phosphate-buffered saline (PBS) at 1:100 and supplemented with 1% dried milk as a blocking agent. Diluted sera were applied to fixed and permeable BCP-1 cells and incubated for 45 min at 22°C. Slides were then washed in blocking buffer, and fluorescein isothiocyanate-conjugated rabbit anti-human immunoglobulin G (IgG) antibody (P0214; DAKO A/S, Denmark), diluted at 1:60 in the blocking solution, was added. Following incubation at 22°C for 22 min, slides were washed in PBS and examined under a fluorescence microscope for nuclear punctate staining (15).
(ii) In-house ELISAs. (a) IMT whole-virus ELISA. The IMT-produced whole-virus ELISA uses a lysate of crude antigens obtained from a mixture of noninduced and TPA-induced BCBL-1 cell lines. Lytic replication was induced by treating BCBL-1 cells with 20 ng/ml of TPA (Sigma) for 5 days. The cell mix was then washed in PBS (pH 7.2) and resuspended in PBS containing sodium deoxycholate at a concentration of 1:50 (based on the initial volume). The supernatant aliquots for viral preparation were obtained after sonication and centrifugation at 3,000 rpm and stored at 80°C. Microtiter plates (Nunc Polysorp) were coated overnight with 100 µl of an optimal dilution of previously titrated antigen (1:100). The plates were washed with PBS containing 0.05% Tween 20 (PBST) and incubated for 60 min with 5% skim milk in PBS; 100 µl of serum diluted 1:200 was dispensed into each well, and the plates were incubated for 1 h at 25°C. After four washings with PBST, anti-human IgG-peroxidase conjugate (P0214; DAKO A/S, Denmark) at 1:4,000 was added to the wells and incubated at 25°C for 40 min. After an additional five washings, the plates were incubated for 10 min at room temperature with a substrate/chromogen (tetramethylbenzidine; Dade Behring). The reaction was stopped and the optical density (OD) read at 450 nm, using 620 nm as a reference wavelength.
Optimal cutoff (CO) values for this assay were evaluated using all serum samples (n = 121) from the main panel, which gave concordant results with four different serological assays (IMT IFA-LANA, IMT IFA-lytic, ABI ELISA, and DIAVIR), i.e., 64 samples were KSHV seropositive and 57 were seronegative by the four assays. The IMT whole-virus ELISA ODs were plotted in a receiver-operator characteristic curve to find the optimal CO maximizing sensitivity and specificity. For a CO of 0.204, the sensitivity and specificity were 97% and 86%, respectively.
(b) MAP ORF K8.1 and 73 ELISA. We modified an ELISA developed and optimized elsewhere (the multiple antigen peptide ELISA [MAP ELISA]) (D. Lagos, unpublished data) for the detection of antibodies against peptide epitopes of LANA-1 (25) and K8.1 (21) synthesized into a tetrameric peptide (two copies of each epitope). Sera were diluted 1:100 in PBST solution and 5% skim milk; 100 µl of diluted serum was dispensed into each well, and the plates were incubated for 1 h at 37°C. After five washings with buffer (Dade Behring), anti-human IgG-peroxidase conjugate (P0214; DAKO A/S, Denmark) at 1:4,000 was added to the wells and incubated at 37°C for 40 min. After four further washings with buffer, 100 µl of color substrate (tetramethylbenzidine; Dade Behring) was added to the wells, and the plates were left in the dark. After 10 min, the reaction was stopped and the OD read at 452 nm, using 620 nm as a reference wavelength. The best CO values for this assay were evaluated using the same 121 serum samples which were used to evaluate the IMT whole-virus ELISA CO values. The MAP ELISA ODs were plotted in a receiver-operator characteristic curve to find the optimal cutoff value to maximize the sensitivity and specificity. At the chosen cutoff value of 0.28, the associated sensitivity was 96%, and the specificity was 98%.
(iii) Commercial ELISAs. (a) HHV-8 IgG antibody ELISA kit. The HHV-8 IgG antibody ELISA kit (Advanced Biotechnologies Inc., Columbia, MD) is based on whole-virus lysates of supernatants obtained from KS-1 cell lines (8). Viral pellets obtained from this supernatant are subjected to centrifugation on a sucrose density gradient to obtain purified virus, and these antigens are used to coat ELISA plates. Testing was performed according to the manufacturer's instructions.
(b) DIAVIR HHV-8 peptide mix ELISA. The DIAVIR test (Biotrim International Ltd., Dublin, Ireland) combines synthetic peptide antigens from viral proteins encoded by ORF K8.1 and ORF 65. Specific antibodies are detected by incubation with anti-human IgG-peroxidase conjugate and by substrate reaction. Testing was performed according to the manufacturer's instructions (34).
Statistical analysis. (i) CRS. We estimated the performance of the IMT and UCL in-house IFA-LANA tests by using a restricted spectrum of sera which included samples from AIDS-KS patients (group 1), representing "likely infected" patients, as well as children (group 7), representing "likely uninfected" patients (panel A; n = 162) populations. We used panel A as a reference standard for comparison of IFA-LANA tests because AIDS-KS patients and children from areas where KSHV infection is not endemic have previously been established as reference groups to assess IFA-LANA performance (16, 23). To expand the spectrum of disease in the validation group, a composite reference standard (CRS) was formed, using the concordant results of the IMT and UCL IFA-LANA tests on the entire panel (panel B; n = 393); all remaining KSHV serological assays were compared against this composite standard. The use of CRS, the process of combining imperfect diagnostic tests and clinical definitions to form an improved comparison standard, has been recommended for the evaluation of new diagnostic assays in the absence of a "gold standard" (1). To avoid incorporation bias, which occurs when the test under evaluation is included as part of the reference standard definition, the performances of IFA-LANA tests produced at UCL and IMT were compared only against a clinical standard (panel A). In comparison with the CRS (panel B), all analyses were performed, excluding equivocal results, and then repeated, considering such results to be negative. We calculated the sensitivity, specificity, and 95% confidence intervals (95% CI) of each KSHV assay compared against either panel A or panel B by using Bayes' theorem (33).
(ii) LCA.
As part of the validation strategy, we used latent class analysis (LCA), an established method of categorical data analysis which has been used previously to validate diagnostic tests in the absence of a perfect reference (20, 32). LCA is ideally suited for this purpose because it makes no assumption about the independence of errors between the diagnostic test and the reference standard, which helps to avoid the known problem of inflated validity estimates encountered with using an imperfect reference standard (38). LCA uses a nonlinear statistical model to classify groups of subjects who have shared characteristics into latent classes (32). It achieves this by maximizing a mathematical likelihood function based on the pattern of responses of input variables (e.g., results of KSHV assays). In this study, KSHV infection was defined as the latent variable, and the results for KSHV serostatus obtained with each test (KSHV infected or uninfected) were the manifest (i.e., input) items. From the input parameters, LCA was used to calculate a posterior probability of the diagnostic class (i.e., probability of KSHV infection given a pattern of results of the test) (32). Equivocal or missing data were excluded from analysis. The goodness of fit of each model was assessed using the Pearson chi-square (
2) test, the likelihood ratio chi-square (L2) test, Akaike information criteria, and Bayesian information criteria (32, 37). We used LatentGold (version 2.0; Statistical Innovations, Belmont, MA) (37) to perform LCA. In order to allow comparability against KSHV serological assays (i.e., a dichotomous KSHV infected/uninfected serostatus), latent classes with similar conditional probabilities (i.e., probability of being KSHV infected given that the test result is positive) for KSHV infection were collapsed into a single category. Because of identical antigen preparations and a "perfect" agreement between IFA-LANA tests from UCL and IMT (kappa = 0.88; P < 0.0001), only the IMT IFA-LANA was included in the final latent class model.
|
|
|---|
Performance of KSHV serological assays against a composite reference standard. As shown in Table 2, both IFA-LANA tests had similar performances, with low sensitivities and nearly perfect specificities, when compared against a clinically defined reference group (panel A). The performances of the five remaining assays varied when compared against the composite standard. The commercial assays (i.e., ABI ELISA and DIAVIR) were shown to have high sensitivities but poor specificities. The results did not materially change after relabeling ABI ELISA and DIAVIR equivocal results as negative (ABI ELISA sensitivity, 98% [95% CI, 93% to 100%]; specificity, 68% [95% CI, 62% to 73%]; DIAVIR sensitivity, 92% [95% CI, 84% to 96%]; specificity, 69% [95% CI, 63% to 74%]). The in-house assays (i.e., IMT IFA-lytic and MAP ELISA) were shown to be the most specific (Table 2).
|
View this table: [in a new window] |
TABLE 2. Performance of KSHV serological assays in comparison with a clinically defined reference standard (panel A) and composite reference standard (panel B)
|
|
View this table: [in a new window] |
TABLE 3. Goodness of fit of latent class models including six KSHV serological assays (IMT IFA-LANA, IMT IFA-lytic, IMT whole-virus ELISA, MAP ELISA, ABI ELISA, and DIAVIR)
|
|
View this table: [in a new window] |
TABLE 4. Conditional probabilities derived from LC model 3, by KSHV assay
|
![]() View larger version (12K): [in a new window] |
FIG. 1. Conditional probabilities of testing KSHV seropositive by six serological assays according to a three-class LCA model. Performances of tests are indicated as class 1 (specificity) and class 2 (sensitivity). The graph shows that samples included in class 1 had low probabilities (0.01 to 0.17) of testing positive by all assays, with IMT IFA-LANA being the test with the highest specificity. In contrast, samples included in class 2 had high probabilities (0.85 to 0.99) of testing positive by all assays, with ABI ELISA being the test with the highest sensitivity.
|
|
View this table: [in a new window] |
TABLE 5 Performances of six KSHV serological assays in comparison with the latent class models presented in and Fig. 1
|
|
|
|---|
As expected, the in-house IFA-LANA tests yielded relatively low sensitivities and high specificities using the clinicoepidemiological reference group. A similar performance has been reported for this assay format among AIDS-KS patients (9). Other assays were able to detect KSHV infection with sensitivities ranging from 85 to 99% and specificities ranging from 65 to 92%, using the panel of sera with assorted probabilities of KSHV infection. Such variations in performance might explain the discrepant results reported in the literature when restricted spectrums of samples are used for assay evaluation. For example, the reported good performance of an ORF K8.1 ELISA (96% sensitivity and 94% specificity) using AIDS-KS patients and healthy blood donors as reference groups (21) dropped to 81% sensitivity and 86% specificity when a broader clinicoepidemiological spectrum of serum samples was included in the study. It is also possible that differences in antigen preparation might account for some of the variation in assay performance in our study. For example, MAP ELISA (ORFs K8.1 and 73) had a higher sensitivity than the IMT IFA-LANA (ORF 73 alone). This improved sensitivity with MAP ELISA could be related to the presence of latent and lytic epitopes in this assay.
Commercially available ELISAs had good sensitivities but poor specificities. These results contrast with the performance of the ABI assay in a small study of 30 AIDS-KS patients and 20 blood donors in the United States, with a reported 80% sensitivity and 100% specificity (36). Improvement in ABI ELISA specificity (>95%) at the expense of sensitivity has been reported by increasing the recommended cutoff points (4, 9, 13). Other reasons for the better performance of the ABI ELISA in other studies could be related to the use of IFA-LANA at a low, 1:20 dilution as the reference test, since low dilutions used with IFA-LANA in earlier studies were later associated with poorer specificity of this test (22).
The poorer specificity of ABI ELISA than of the IMT whole-virus ELISA, using both the CRS (65% versus 72%) and LCA (83% versus 100%), is noteworthy. Although the assays are based on the same viral target (whole virus), the observed differences might be related to antigen preparation: the ABI ELISA uses a purified virion lysate obtained from supernatant fluids from the KS-1 cell line, whereas the IMT whole-virus ELISA uses a lysate of crude antigens obtained from a mixture of noninduced and TPA-induced BCBL-1 cell lines. We did not retest equivocal results found with ABI ELISA (n = 29 [6.5%]) and DIAVIR (n = 21 [4.7%]). However, the great majority of equivocal results (28/29 for ABI ELISA and 19/21 for DIAVIR) were concentrated in groups at intermediate or low risk of KSHV infection (groups 3 to 7), and the performances of these assays did not materially change when equivocal results were eliminated or considered negative in the analysis.
We used LCA to evaluate the performances of KSHV serological assays because conventional contingency two-by-two tables might not provide accurate estimations of a test performance when no "gold standard" is available (17). LCA has already been used to assess KSHV serological assays in U.S. (27), African, and Mediterranean populations (12). In one study (12), the final LCA model indicated that the ORF K8.1 ELISA had a higher sensitivity and specificity (100% and 98%, respectively) than did ORF 65 ELISA (76% and 57%, respectively) and IFA-LANA (88% and 69%, respectively). However, no clinical or serological reference standards were used in that study to confirm the results of the LCA model. In our study, LCA provided results consistent with those obtained by conventional Bayesian methods. Firstly, the in-house IFA-LANA tests were shown to be the least sensitive, although most specific, of all assays. Secondly, the poor performances of commercial assays were confirmed. Finally, LCA showed that the IMT whole-virus ELISA had the best all-around performance. This assay thus represents a valuable alternative to commercial diagnostic tests, particularly when large numbers of samples need to be screened in resource-constrained environments. We are confident that the comparability of results between CRS and LCA strengthens the validity of our findings, since the mechanisms by which the reference standards were derived were distinct: the CRS was based on a summative approach, while LCA takes into account information from all six evaluated tests, using maximum likelihood estimation.
In summary, the lack of a "gold standard" and past performance discrepancies reported with previous KSHV serological assays prompted this thorough evaluation of available in-house and commercial tests. We recommend that testing algorithms to detect KSHV antibodies in populations should include a careful evaluation of available serological assays, using a wide spectrum of serum samples.
We thank David Mabey and Onno Dekker (LSHTM) for their support with the study, Chris Boshoff (University College London, UCL) and Helen Weiss (LSHTM) for scientific advice, Dimitrios Lagos (UCL) for antigen preparations (MAP ELISA) and useful comments, and Dimitra Bourboulia (UCL) and Silvia Franceschi (International Agency for Research on Cancer) for useful comments and critical reviews of this paper.
Published ahead of print on 20 December 2006. ![]()
|
|
|---|
| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Copyright © 2009 by the American Society for Microbiology. For an alternate route to Journals.ASM.org, visit: http://intl-journals.asm.org | More Info»