ABSTRACT
In this study, we aimed to investigate the performance of nontreponemal antibody tests in cerebrospinal fluid (CSF) specimens from syphilis patients. From September 2009 to September 2012, CSF specimens were collected at the Shanghai Skin Disease Hospital in Shanghai, China, from 1,132 syphilis patients without HIV infection, including 154 with symptomatic and 56 with asymptomatic neurosyphilis. All of the CSF specimens underwent testing with a rapid plasma reagin (RPR) test, an RPR-V (commercial RPR antigen diluted 1:2 in 10% saline) test, the toluidine red unheated serum test (TRUST), and the Venereal Disease Research Laboratory (VDRL) test. Specificities, sensitivities, positive predictive values (PPVs), negative predictive values (NPVs), and kappa values were calculated to determine the performances of the tests. We compared results of the CSF-VDRL, CSF-RPR, CSF-RPR-V, and CSF-TRUST among patients with symptomatic and asymptomatic neurosyphilis who had reactive CSF-Treponema pallidum particle agglutination (TPPA) test results. Overall, the CSF-VDRL test was reactive in 261 patients (23.1%). There were no cases in which the CSF-VDRL was nonreactive and CSF-RPR, CSF-RPR-V, or CSF-TRUST was reactive. Agreement between the results of CSF-TRUST and CSF-RPR was almost perfect (κ = 0.861), with substantial agreement between the results of CSF-RPR and CSF-RPR-V (κ = 0.740). The sensitivities of CSF-VDRL, CSF-RPR, CSF-RPR-V, and CSF-TRUST were 81.4%, 76.2%, 79.5%, and 76.2%, respectively. Compared to CSF-VDRL, CSF-RPR, CSF-RPR-V, and CSF-TRUST had comparable PPVs and NPVs. However, the specificity of CSF-VDRL (90.3%) was significantly lower than those of the other tests (92.7 to 93.4%). Therefore, CSF-RPR, CSF-RPR-V, and CSF-TRUST can be considered alternative tests for neurosyphilis diagnosis in HIV-negative populations, particularly when the CSF-VDRL is not available.
INTRODUCTION
Treponema pallidum subsp. pallidum, the pathogen of syphilis, can disseminate into the central nervous system (CNS) within days after exposure (1). Neuroinvasion with T. pallidum subsp. pallidum can lead to asymptomatic meningitis in approximately 14% to 20% of cases and, if untreated, can progress to severe and irreversible symptomatic neurosyphilis with cerebrospinal fluid (CSF) abnormalities (2). Early diagnosis of neurosyphilis is critical for timely treatment and minimization of sequelae.
The CSF-Venereal Disease Research Laboratory test (VDRL) is currently considered the standard test for confirming a neurosyphilis diagnosis (3, 4). However, several limitations have been noted with the test, including its moderate sensitivity, lack of commercial availability in resource-limited countries, and cumbersome and time-consuming procedures.
Compared to VDRL, the rapid plasma reagin test (RPR) and the toluidine red unheated serum test (TRUST) have similar test principles and performance using serum specimens, are easier to perform, and are available as commercial test kits (5, 6). RPR and TRUST may be promising alternatives to VDRL when testing CSF for neurosyphilis; however, their use in routine clinical practice is currently controversial because of limited experience and published reports in the literature. Therefore, we conducted this study in order to compare the performance of CSF-VDRL, CSF-RPR, and CSF-TRUST for the diagnosis of neurosyphilis, using a large sample of patients identified with syphilis in China.
MATERIALS AND METHODS
Study participants.Between September 2009 and September 2012, we recruited eligible patients diagnosed with syphilis prior to therapy who presented to the Sexually Transmitted Disease Institute at the Shanghai Skin Disease Hospital in Shanghai, China. The inclusion criteria for potential subjects included age >18 years old, new syphilis diagnosis regardless of clinical stage (primary, secondary, or latent syphilis or suspected neurosyphilis), or serofast status, which was defined as having a <4-fold decline in titer after therapy and having a constant low RPR titer of ≤1:8 despite receiving standard syphilis treatment and having at least 2 years of follow-up evaluation (7). In China, the standard treatment for early or late syphilis (without neurosyphilis) consists of benzathine penicillin 2.4 MU administered as intramuscular injections every week for 3 weeks. We excluded syphilis patients if they were less than 18 years old, pregnant, or coinfected with HIV and if they refused lumbar puncture for evaluation as part of the study procedure. After obtaining written informed consent, we interviewed each participant for a detailed medical history and performed a routine neurologic examination for assessment of cranial nerve function, motor function, sensation, coordination, reflexes, and gait. Venous blood and CSF were collected from all subjects for syphilis testing and evaluation per standard clinic practice at the hospital in Shanghai. All the participants also underwent HIV antibody testing. This study was approved by the ethics committee of the Shanghai Skin Disease Hospital.
Laboratory tests.For syphilis diagnosis, the serum RPR was performed as the screening test, followed by the T. pallidum subsp. pallidum particle agglutination test (TPPA) for confirmation (7). Routine testing of CSF was conducted to examine the inflammatory responses in the CNS, including the CSF white blood cell (WBC) count, red blood cell (RBC) count, and total protein. The CSF samples underwent testing with CSF-VDRL, CSF-RPR, and CSF-TRUST for neurosyphilis diagnosis, using the same procedures applied to serum specimens. Furthermore, we diluted commercial RPR antigen 1:2 in 10% saline, allowed it to stand for 5 min before use, and named it CSF-RPR-V as described in a previous study (8). All the tests described above were conducted using the same thawed CSF aliquot from each participant on the same day by the same operator, who was masked to the patient's clinical status. All the laboratory tests were performed in the central laboratory of the Shanghai Skin Disease Hospital according to the standard methods.
Since we were comparing the test performance of the CSF-VDRL with CSF-RPR, CSF-RPR-V, and CSF-TRUST, we were not able to use the CSF-VDRL as the reference standard for neurosyphilis diagnosis. Rather, we used CSF-TPPA as a marker in the diagnosis of neurosyphilis (9, 10, 11), which was internally validated as a CSF treponemal antibody-based assay in our laboratory using CSF-fluorescent treponemal antibody absorption test (FTA-ABS) for comparison (unpublished data). The test performance of CSF-TPPA is statistically equivalent to that of CSF-FTA-ABS for neurosyphilis diagnosis (κ = 0.953, P < 0.001) (see the supplemental material).
Case definitions.The diagnosis and clinical stage of syphilis were determined based on updated sexually transmitted disease surveillance case definitions (12). Symptomatic neurosyphilis was defined as the combination of clinical symptoms or signs consistent with neurosyphilis without other known causes of the clinical abnormalities, with a positive CSF-TPPA in the absence of contamination with blood. Asymptomatic neurosyphilis was defined as the combination of elevated CSF WBC count (≥10/μl) without other known causes, with a positive CSF-TPPA in the absence of contamination with blood. Participants who had only a positive CSF-TPPA result in the absence of symptoms or CSF pleocytosis were not considered to have neurosyphilis.
Statistical analysis.The Statistical Package for the Social Sciences for Windows (SPSS, version 13.0; Chicago, IL) was used for statistical analysis. Descriptive statistics were used to calculate the median and interquartile range (IQR). Sensitivities, specificities, positive predictive values (PPVs), negative predictive values (NPVs), and kappa values (κ) were calculated using standard formulae. The agreement of the results by κ value was categorized as almost perfect (0.81 to 1.0), substantial (0.61 to 0.80), moderate (0.41 to 0.60), fair (0.21 to 0.40), and slight (0.00 to 0.20). Additionally, the chi-square test was performed to compare the proportion between groups. Differences were considered to be statistically significant at two-sided P values of <0.05.
RESULTS
Participant characteristics.A total of 1,132 eligible participants were enrolled in the study prior to therapy with either primary, secondary, or latent syphilis, suspected neurosyphilis, or serofast status (Fig. 1). Among them, 154 and 56 participants were diagnosed with symptomatic neurosyphilis and asymptomatic neurosyphilis, respectively, using the definitions above; four participants with neurologic symptoms and signs but negative results of the CSF-TPPA were excluded from the analyses. The detailed characteristics of the study participants are shown in Table 1 and Table 2.
Flow diagram of participant enrollment.
Characteristics of all study participants
Characteristics of the participants diagnosed with asymptomatic or symptomatic neurosyphilis
CSF-VDRL, CSF-RPR, CSF-RPR-V, and CSF-TRUST results.The CSF-VDRL was reactive in 261 patients, among which the number of samples that were also reactive by CSF-RPR, CSF-RPR-V, and CSF-TRUST were 217, 225, and 220, respectively. Among the 261 specimens with positive CSF-VDRL results, the CSF-RPR, CSF-RPR-V and CSF-TRUST results were negative among 16.9%, 13.8%, and 15.3%, respectively. There were no specimens for which the CSF-VDRL was nonreactive but CSF-RPR, CSF-RPR-V, or CSF-TRUST was reactive. Three samples were reactive by CSF-RPR but nonreactive by CSF-TRUST, and seven samples were reactive by CSF-TRUST but nonreactive by CSF-RPR. Among the 208 samples that were reactive in all four tests, the median serum RPR titers (1:4) and IQRs (1:2 to 1:8) were statistically identical.
Among patients with symptomatic neurosyphilis, the κ values between the CSF-VDRL and the other CSF nontreponemal tests were all substantial, at 0.709. Among the patients with asymptomatic neurosyphilis, the κ values were 0.692 between CSF-VDRL and CSF-RPR or CSF-TRUST and 0.781 between the CSF-VDRL and CSF-RPR-V. Among patients with positive CSF-VDRL results, the agreement between the results of CSF-TRUST and CSF-RPR was almost perfect (κ = 0.861), and there was substantial agreement between the results of CSF-RPR and CSF-RPR-V (κ = 0.740) (Table 3).
Comparisons between CSF-RPR and CSF-RPR-V or CSF-TRUST results among 261 participants with positive results of the CSF-VDRL
Table 4 shows the overall sensitivities, specificities, PPVs, and NPVs of the four CSF nontreponemal tests among participants based on our definitions of neurosyphilis. CSF-VDRL and CSF-RPR-V had the highest sensitivities; however, there were no statistical differences in sensitivities among the four tests. The specificity of CSF-VDRL was significantly lower than that of CSF-RPR (90.3% versus 93.4%, P = 0.017) and CSF-TRUST (90.3% versus 93.1%, P = 0.028). The PPVs and NPVs of those four tests were not statistically different (P > 0.05).
Sensitivities, specificities, positive predictive values, and negative predict values of the four CSF nontreponemal tests for neurosyphilis diagnosisa
When we distinguished between symptomatic and asymptomatic neurosyphilis, we found that the four CSF nontreponemal tests had better test performances for diagnosing symptomatic neurosyphilis than did asymptomatic neurosyphilis (Table 5). For diagnoses of symptomatic neurosyphilis, the sensitivities, PPVs, and NPVs of the four tests were virtually identical; however, the specificity of the CSF-VDRL was significantly lower than that of CSF-RPR (86.7% versus 90.2%, P = 0.019) and the CSF-TRUST (86.7% versus 90.1%, P = 0.023). For diagnosing asymptomatic neurosyphilis, we found no statistical differences in the sensitivities, specificities, PPVs, and NPVs of the four tests.
Sensitivities, specificities, positive predictive values, and negative predict values of the four CSF nontreponemal tests for symptomatic and asymptomatic neurosyphilis diagnosis
We also compared the characteristics of the participants with inconsistent CSF-VDRL, CSF-RPR, CSF-RPR-V, or CSF-TRUST results. The CSF WBC count was statistically higher among patients who were CSF-VDRL positive (CSF-VDRL+)/CSF-RPR+ and CSF-VDRL+/CSF-TRUST+ than those with inconsistent results. The CSF protein was also statistically higher among patients who were CSF-VDRL+/CSF-RPR+, CSF-VDRL+/CSF-RPR-V+, and CSF-VDRL+/CSF-TRUST+ than those with inconsistent results (Table 6). However, these differences may not be meaningful from a clinical standpoint.
Comparison of the characteristics of the participants with inconsistent results between CSF-VDRL and the CSF-RPR, CSF-RPR-V, or CSF-TRUST
DISCUSSION
The CSF-VDRL has a high specificity and is currently considered the definitive test for diagnosis of neurosyphilis globally. However, the reagent needs to be prepared and must be used within 2 h, and a light microscope is required for detection (13). Therefore, there is a need to identify simpler, more convenient and more efficient methods as alternative CSF assays. We analyzed data from the largest study to date involving 210 neurosyphilis patients and found that for diagnosis of neurosyphilis based on our definitions, CSF-RPR, CSF-RPR-V, and CSF-TRUST had comparable sensitivities (76.2 to 79.5%) and higher specificities (92.7 to 93.4%) than the CSF-VDRL (sensitivity 81.4%, specificity 90.3%).
Our findings are consistent with the reports from Castro et al. (5) regarding the performance of CSF-RPR and Jiang et al. regarding the CSF-TRUST, which involved 24 and 41 neurosyphilis patients, respectively (6). In an earlier study in 1985, Larsen et al. concluded that CSF-RPR and CSF-TRUST had statistically significantly lower sensitivities and specificities than CSF-VDRL; however, their study was limited by a small sample of patients and older diagnostic techniques (14).
A more recent study involving 72 patients with neurosyphilis, Marra et al. reported that CSF-RPR and CSF-RPR-V had lower sensitivities of 51.5% and 57.6%, respectively, compared to 66.7% for the CSF-VDRL for symptomatic neurosyphilis (8). They also stated that CSF-RPR had a high false-negative rate of 35.6%; therefore, CSF-RPR was not recommended as an alternative for CSF-VDRL. However, we found that the false-negative values of CSF-RPR, CSF-RPR-V, and CSF-TRUST were 16.9%, 13.8%, and 15.3%, respectively, compared to the CSF-VDRL. The differences in these findings may be due to the HIV status of the study populations and the definitions for neurosyphilis. In the study by Marra et al., most of the syphilis patients were coinfected with HIV, and in these patients the investigators defined a CSF WBC count of >20/μl as one of the diagnostic criteria for neurosyphilis (8). In comparison, all of our study participants were HIV negative and we used a CSF WBC count of ≥10/μl according to the updated CDC guidelines in China (7). Another study comparing the CSF-TRUST and CSF-VDRL among an HIV-negative population with neurosyphilis was conducted by Gu et al. (15). However, they did not provide clinical data, making it difficult to differentiate syphilis patients who had received therapy from those who received initial treatment and to make further comparisons with our study.
We found that CSF-RPR-V had better agreement (κ = 0.781) with the CSF-VDRL than CSF-RPR (κ = 0.692). CSF-RPR-V also resulted in a slightly higher sensitivity for neurosyphilis diagnosis than CSF-RPR and CSF-TRUST, although these results were not statistically significant. Therefore, if CSF-VDRL is not available and there are resources to conduct the additional steps to dilute RPR antigen 1:2 in 10% saline, CSF-RPR-V should be considered an alternative test for neurosyphilis diagnosis.
Some limitations of this study should be acknowledged. First, it is generally known that evaluation of diagnostic tests varies based on definitions of the gold standard. There is actually no diagnostic gold standard for neurosyphilis, although the CSF-VDRL has been widely used in clinical practice. Since we aimed to compare the diagnostic performance of CSF-VDRL with CSF-RPR, CSF RPR-V, and CSF-TRUST, we could not use CSF-VDRL as a reference standard. Rather, as a marker to assist in our definitions of symptomatic and asymptomatic neurosyphilis, we used the CSF-TPPA, which is recommended as one of the diagnostic tests for neurosyphilis in the European guidelines (9). In our study, we found no cases in which the CSF-VDRL was reactive but the CSF-TPPA was nonreactive, supporting the feasibility of using the CSF-TPPA since no confirmed neurosyphilis cases were missed. Second, we did not include syphilis patients coinfected with HIV, since they may have impaired antibody responses to the antigen used in VDRL, RPR, and TRUST on CSF. Any generalization of the results from this study to HIV-positive or other immunocompromised patients should therefore be made with caution. Lastly, we found no statistical difference in sensitivities between the four CSF nontreponemal tests, which may have been due to our sample size of participants with neurosyphilis. Although we analyzed data from the largest study to date, involving 210 neurosyphilis patients, this number may still not have been sufficient to detect significant differences in sensitivities between the tests.
Syphilis is now resurgent with a vengeance in China, the largest developing country globally (16), and the incidence of neurosyphilis may increase accordingly. Since the CSF-VDRL is not widely available in all hospitals, because of the limitations mentioned above, this precludes the early diagnosis and timely treatment of neurosyphilis. We found that CSF-RPR, CSF RPR-V, and CSF-TRUST (which are attractive since the commercial test kits for TRUST and RPR are widely used in China) have sensitivities, specificities, PPVs, and NPVs comparable to those of the CSF-VDRL. Furthermore, conducting CSF-RPR or CSF-TRUST is less expensive than conducting the CSF-VDRL (15 yuan per test versus 90 yuan per test). Therefore, we recommend that CSF-RPR, CSF RPR-V, or CSF-TRUST be considered as alternative tests for neurosyphilis diagnosis in HIV-negative populations. These alternative CSF nontreponemal tests will result in false-negative rates that range between 14 and 17%, but are preferable to having no test at all for neurosyphilis diagnosis in areas where the CSF-VDRL is not available.
ACKNOWLEDGMENTS
This work was supported by grants from the National Science Foundation of China (30972664), the Basic Research Project of Shanghai Science and Technology Commission (11JC1411600), the Shanghai Health Bureau Foundation (2010Y137 and 20114026), and the Shanghai Hospital Development Center Foundation (12012237).
We greatly appreciate the clinic staff who recruited patients and we thank all participants of this study for their cooperation. We also thank Fred Sparling from the University of North Carolina School of Medicine and John Saunders from St Bartholomew's Hospital for critical comments on the manuscript.
We declare that we have no conflicts of interest.
FOOTNOTES
- Received 14 September 2013.
- Returned for modification 1 October 2013.
- Accepted 7 December 2013.
- Accepted manuscript posted online 11 December 2013.
Supplemental material for this article may be found at http://dx.doi.org/10.1128/JCM.02522-13.
- Copyright © 2014, American Society for Microbiology. All Rights Reserved.
