![[Feedback]](/icons/banner/feedbackACT.gif){kind=icon}
![[For Subscribers]](/icons/banner/subscriberACT.gif){kind=icon}
![[Archive]](/icons/banner/archiveACT.gif){kind=icon}
![[Search]](/icons/banner/searchACT.gif){kind=icon}
![[Contents]](/icons/banner/tocACT.gif){kind=icon}
Previous Article | Next Article 
Journal of Clinical Microbiology, July 2006, p. 2512-2517, Vol. 44, No. 7
0095-1137/06/$08.00+0 doi:10.1128/JCM.02620-05
Copyright © 2006, American Society for Microbiology. All Rights Reserved.
Chlamydia Research Laboratory, Department of Laboratory Medicine, University of California, San Francisco, California,1 Sexually Transmitted Disease Control Branch, California Department of Health Services, Richmond, California2
Received 16 December 2005/ Returned for modification 6 February 2006/ Accepted 6 May 2006
 |
ABSTRACT |
|---|
 Top Abstract IntroductionMaterials and MethodsResultsDiscussionReferences |
|---|
|
INTRODUCTION |
|---|
|
TopAbstractIntroductionMaterials and MethodsResultsDiscussionReferences |
|---|
Package insert specificities range from 97.6 to 98.9% for these tests. However, these specificities are not based on discrepant analysis, because statisticians have criticized that method (6, 16, 19). The true specificities of NAAT methods are in fact much higher, and several studies have found specificities to be >99% (1, 4, 8, 22). With a wide range of reported specificities and no guidance from the CDC, laboratories must decide which specificities are valid. If product insert specificities are accepted, then by CDC recommendations, all positive specimens in the low- to moderate-prevalence range (1 to 7%) would need confirmatory testing because PPVs are <90%.
The CDC has suggested several possible strategies for confirmation, but it is not clear which approach is best. The proposed strategies include (i) testing a second specimen with a different test or target, (ii) performing a second NAAT targeting a different nucleic acid sequence on the original specimen, (iii) repeating the original test on the original specimen, and (iv) bringing the patient back for a retest. These approaches have limitations: (i) most clinicians will not routinely collect two specimens for the same evaluation, nor is it feasible to bring the patient back to collect another sample, and (ii) most laboratories do not have the capability to perform two different NAAT methodologies. Currently, the approach of repeat testing is the only feasible confirmatory test available to most laboratories.
The concept of confirmatory testing is not new (2, 5, 12). In February 2001, following reports of low test performance and quality control issues with their test (5), Abbott Laboratories suggested that all positive patient samples in their LCx Probe System Chlamydia trachomatis Assay should be confirmed by a repeat test. Eventually, the product was discontinued, but many laboratories switched to other NAATs and continued to retest positive results as part of their standard operating procedure.
From 2001 to 2004, we performed confirmatory nucleic acid amplification testing, using different strategies. Here, we report our experiences when confirming positive results for C. trachomatis by repeating the original test on the original specimen, by testing the original specimen with a different NAAT, and by testing duplicate specimens.
|
MATERIALS AND METHODS |
|---|
|
TopAbstractIntroductionMaterials and MethodsResultsDiscussionReferences |
|---|
Specimen types. For women, we tested first-catch urine (FCU) and endocervical swabs. For men, we tested FCU and urethral swabs. For men who had sex with men, we also tested rectal and pharyngeal swabs, specimens not FDA cleared for NAATs. All specimens were initially tested by a NAAT for C. trachomatis, and positive specimens were saved for confirmatory testing. If retested within 48 h, specimens were stored at 4°C. If not, then the specimens were stored at –70°C.
Approach 1: repeat testing for confirmation of positive results. Here and elsewhere, where a subset analysis is mentioned, there was neither randomization nor specific selection criteria (see below for reasons) for specimens for a retest.
(i) LCx. From February 2001 to November 2002, routine clinical specimens were initially tested by LCx. All positive specimens were repeat tested by LCx. On a subset of these positive results, the original LCx samples were confirmed by PCR (250 µl of the LCx sample was spun at 12,000 x g for 15 min, the supernatant was discarded, and the protocol for PCR swab processing and testing was followed). In addition, LCx positive specimens with initial low-level (>1.0 but <1.9) sample rate/cutoff ratios were retested repeatedly (eight times) in different LCx assay runs.
AC2. From June 2003 to July 2004, clinical specimens were initially tested by the AC2 test. All positive specimens were repeat tested by AC2. The original positive specimens were then confirmed by the ACT assay. In addition, AC2 positive specimens with initial low-level readings were retested on six separate assay runs.
(ii) PCR. From June 2003 to March 2004, some clinical specimens were initially tested by PCR, and positive specimens were repeat tested by PCR.
(iii) SDA. From June 2003 to August 2004, some clinical specimens were initially tested by SDA. Positive specimens were repeat tested by SDA twice. In addition, positive specimens with initial low-level method other than acceleration (mota) readings were retested on four separate assay runs.
(iv) Repeat testing for specificity. With each assay (LCx, SDA, and AC2), a subset of specimens with an initial negative C. trachomatis result was retested repeatedly by the same assay in different assay runs.
Approach 2: use of a different NAAT for confirmation. (i) Study 1. From January 2003 to September 2004, a moderate-prevalence population was tested for C. trachomatis. Swabs and FCU were collected from men and women seen in obstetrics/gynecology (OB/GYN) clinics, family planning clinics, or a sexually transmitted disease (STD) clinic in the San Francisco Bay area. Specimens were collected in BD's ProbeTec transport tubes and originally tested by SDA. C. trachomatis-positive specimens were repeat tested by SDA, and then aliquots of the original specimens were tested by PCR and AC2. For PCR confirmation, 500 µl of the SDA sample was spun at 12,000 x g for 15 min, the supernatant was discarded, and the protocol for PCR swab processing and testing was followed. For AC2 confirmation, 100 µl of the SDA sample was inoculated into an AC2 swab transport tube and tested according to the package insert instructions.
(ii) Study 2. From May 2003 to June 2004, a low-prevalence population was screened for C. trachomatis. Swabs and FCU were collected from women seen in OB/GYN clinics, family planning clinics, and an HMO—Kaiser Permanente Group in Southern California. The great majority of the patients were asymptomatic and >21 years old. Specimens were originally tested by SDA. All positive specimens were held at –70°C until they were shipped to the University of California, San Francisco, Chlamydia Research Laboratory. Upon receipt, the samples were thawed and retested by SDA, as well as PCR, AC2, and ACT. For PCR, AC2, and ACT confirmation, the SDA sample was treated as described above. A separate 400-µl aliquot was tested with the AC2 and ACT assays.
(iii) Study 3. From October 2003 to June 2004, a low-prevalence population for C. trachomatis was screened in OB/GYN clinics in southern California. The majority of patients were asymptomatic. Cervical brushes were collected in PreservCyt (ThinPrep Pap Test; Cytyc Corp., Boxborough, MA) and tested by HC2 for C. trachomatis. All positive specimens were held at –70°C until they were shipped to the University of California, San Francisco, Chlamydia Research Laboratory. Upon receipt, the samples were thawed and tested by SDA, PCR, AC2, and ACT. For SDA confirmation, 4.0 ml of the positive PreservCyt was spun at 2,000 x g for 30 min, the supernatant was discarded, and the protocol for swab processing and testing was followed. For PCR confirmation, 500 µl of the positive PreservCyt was spun at 12,000 x g for 15 min, the supernatant was discarded, and the protocol for swab processing and testing was followed. For AC2 and ACT confirmation, 100 µl of the positive PreservCyt was inoculated into an AC2 swab transport tube. A separate 400-µl aliquot was tested with the AC2 and ACT assays.
Approach 3: testing a second specimen with a different test or target for confirmation. The study using approach 3 was recently published (17). Briefly, from January 2003 to May 2003, a multicenter clinical evaluation of the AC2 and ACT assays was conducted in North America. Subjects attended STD, OB/GYN, teen, or family planning clinics in seven geographically diverse centers. There were 1,465 females and 1,322 males enrolled. FCU and two randomized endocervical swabs were collected from females. FCU and two randomized urethral swabs were collected from males. Each specimen type was tested by ACT, AC2, and SDA. Thus, three NAATs, each having a different target, were used to test specimens for each subject. The SDA, ACT, and AC2 assays were each considered the primary screening tests, and all positive results for each test were then confirmed by both of the other tests.
|
RESULTS |
|---|
|
TopAbstractIntroductionMaterials and MethodsResultsDiscussionReferences |
|---|
5% for the reproducibility studies. AC2 and PCR repeated 97.7% and 96.7% of positive results, respectively. SDA confirmed only 83.8% by repeat testing. However, by testing SDA positive specimens a third time, an additional 9% were confirmed. LCx high-level positive results seldom failed to repeat; failure to retest positive was associated with specimens having low (1.0 to 1.9) sample rate/cutoff ratios, where 53% of the positive results did not repeat (Fig. 1). Similarly, 58% of low-level SDA positive (2,000 to 10,000) mota readings did not repeat, but failures were seen in all ranges, even at the high end of >40,000 mota (Fig. 2). When low-level LCx, SDA, and AC2 positive specimens were retested several times, variable results were observed (Table 2). Each positive result was ultimately confirmed, but an apparently random sequence of positive and negative results was seen with the retests. In two of these repeat studies (LCx and AC2), we were also able to use another NAAT for confirmation. Using PCR on LCx positive specimens failed to confirm some specimens that repeat testing confirmed (Table 1). By using ACT to confirm AC2 (Table 1), confirmation was slightly better than by repeat testing (99.1% versus 97.7%). Of the AC2 repeat failures, four of five samples had low relative light unit readings (
500,000).
|
|
|
|
|
21% were male. The overall prevalence of C. trachomatis was 6.7%. There were a total of 130 SDA C. trachomatis original positive results (Table 4), of which 87.7% repeat tested as positive. When different NAATs were used for confirmation, PCR confirmed 83.8% while AC2 confirmed 86.9%. However, by using a combination of repeat SDA, PCR, or AC2, 94.6% of the original positive results were confirmed. For study 2, we screened 4,961 patients with an overall C. trachomatis prevalence of 3.5%. There were a total of 111 SDA C. trachomatis original positive results (Table 4). We found that only 79.3% repeat tested positive (P = 0.077) compared to study 1. When different NAATs were used for confirmation, PCR confirmed 73.9%, while AC2 and ACT both confirmed 81.1%. However, by using different combinations of confirmatory tests, more positive results (90.1%) were confirmed.
|
Table 5 shows our results using either a second specimen with a different NAAT for confirmation or a second NAAT on the same specimen (17). For all specimen types, AC2 had 850 positive results, the ACT assay had 927 positive results, and the SDA had 720 positive results. Both the ACT and AC2 confirmed 96.9% of the SDA positive results. Of the AC2 positive specimens, 98.1% were positive by ACT, but only 82% were positive by SDA. Of the ACT positive results, AC2 confirmed 89.8%, but the SDA test confirmed only 75.1%. There were no major differences in the observed patterns by sex or specimen type. SDA consistently confirmed fewer of the initially positive AC2 and ACT results in a second specimen or in the same FCU specimen.
|
|
DISCUSSION |
|---|
|
TopAbstractIntroductionMaterials and MethodsResultsDiscussionReferences |
|---|
It would seem, then, that much of the problem with reproducibility of positive NAAT results actually reflects the exquisite analytic sensitivity of these assays. It would be useful if there was an accepted international Chlamydia standard that laboratories could use to perform interassay evaluations and, through dilution studies, better understand the vagaries of testing near the assay detection limits. The enhanced analytic sensitivity of the NAATs also means that minor variations in processing could impact results. More emphasis should be placed on quality control and proficiency testing. Specimen adequacy could be an issue, and we might benefit from research on the possible benefits of including assays for human housekeeping genes in these diagnostic tests.
The reproducibility of C. trachomatis-negative NAAT specimens was evaluated. Table 3 shows high reproducibility of negative results for LCx, SDA, and AC2, all >99.5%. These results are similar to C. trachomatis NAAT specificities found in many clinical evaluations using discrepant analysis. They are higher than the specificities listed in the package insert.
The CDC-suggested approach of performing a second NAAT targeting a different nucleic acid sequence on the original specimen for C. trachomatis confirmation also varies in performance. Although NAATs are highly sensitive, there are differences among them. Some are more sensitive than others. For both SDA confirmation studies using a different test, PCR confirmed fewer positives than repeat testing, whereas AC2 confirmation was similar to that of repeat testing. With a moderate C. trachomatis prevalence, AC2 confirmed 86.9% of SDA positive results (Table 4). However, in evaluation of a different NAAT with a second specimen, we found that AC2 confirmed 96.9% of the SDA positive results (Table 5). Here, the higher number of positives confirmed was probably due to the methodology, as the collection of a second specimen in the recommended NAAT transport medium reflects more optimal testing conditions. Although our procedure using the original SDA transport medium was validated with AC2, ACT, and PCR, it may not be the most sensitive. PreservCyt transport medium was used for HC2 specimens, and all the NAATs had similar confirmation rates of 90%. By using a combination of three NAAT results, 93.1% of HC2 positive results were confirmed.
In the evaluation of the third approach, it was clear that some NAATs cannot be used to confirm positive results from other NAATs because of differences in sensitivity (17). The use of the less sensitive SDA to confirm AC2 or ACT positive results would have resulted in wrongly reporting 15% of the confirmable ACT results as negative (i.e., not confirmed). When the SDA assay was used for the primary screening, both the AC2 and ACT assays confirmed 96.9% of the results. Thus, when attempting to confirm positive NAAT results, the assays employed should have equivalent sensitivities.
We have shown that all three of the CDC-suggested approaches for confirmation vary in performance. The use of just one approach would fail to confirm many positives. By using two of the approaches (repeat testing with the same test and with a different NAAT), we found 90.1 to 94.6% of SDA positive results were confirmed and 99.5% of AC2 positive results were confirmed. With >90% of our original C. trachomatis-positive results being confirmed in all the assays, we found that confirmatory testing of positive NAAT specimens is not necessary. This does not imply that false-positive results do not occur. They do, but at a very low rate, and confirmatory testing simply is not the way to identify them.
The logic behind confirmatory testing is based upon two assumptions. The first is that if a positive result is repeated, it is likely to be a true positive. The underlying assumption here is that false-positive results will not be confirmed. It is unclear why this should be true, particularly when the same test or target is used. We do not have sufficient experience with proven false-positive specimens to evaluate this assumption, but logic says that it need not be true; if there is a compound capable of causing a false-positive result in the specimen, why should it not be there for the retest?
The second assumption is that failure to repeat an initial positive result means the initial result was incorrect (i.e., a false positive). That assumption is demonstrably false. We found that the set of specimens in which an initial result was positive and a second test was negative had a higher prevalence of positive results on further testing than any incoming set of specimens (i.e., approximately 65% of these specimens could be shown to be positive in the third or subsequent tests). It is clear that some of these results reflect heterogeneity of these specimens (i.e., consistent with a Poisson distribution, where a single target is the threshold level of analytic sensitivity for the specific test). If on average the amount of target present in that specimen is at that level, then a large fraction (as high as 35%) of aliquots from the specimen would test negative. Certainly, we saw something like that happening with both clinical specimens and seeded specimens (Table 2). Thus, the initial positive result most likely reflects truth (i.e., the target was present in the original specimen, and the failure to verify was because there was too little target homogeneously distributed in the specimen to yield positive results in every subsequent test).
These results have implications beyond testing for Chlamydia trachomatis, or for that matter, Neisseria gonorrhoeae (for which we had a similar experience with confirmatory testing with AC2 and SDA, but not with PCR). There will probably be similar results with efforts to confirm positive results in NAATs for other organisms. There are other implications. The variations we saw in test results are probably reproduced when multiple specimens are collected at different times from individuals with low-level infections. In other words, sampling over different time periods could result in intermittent negatives, and that may be misinterpreted as reflecting spontaneous clearance of infection. Rather than clearing of the infection, the negative result may reflect sampling variability or simply low target levels (in the NAAT sense). Years ago, when we evaluated the sensitivity of culture and did not have a more sensitive assay available, we estimated that the sensitivity of culture was on the order of 80% based on repeat testing of individuals prior to their being treated (14). Individuals who initially tested positive would test negative at the second visit and then positive at a third visit. Most of these data were developed during prospective studies of the risk of infection to newborns passing through an infected birth canal. The pregnant women were being seen on a regular basis, and those who were infected were often not found to be positive in isolation attempts during each visit. We assumed that with the use of the more sensitive NAATs, this sporadic positivity seen with culture would not be seen in these more sensitive assays. Our results suggest that is not likely to be the case. Thus, the failure to repeat positive results when rescreening individuals should not easily be accepted as spontaneous clearance; it could reflect the variability inherent with low copy numbers of targets.
There is another implication of our results. If low-level positive results give variable results on repeat testing, it implies that some positive specimens will give negative results on the first test. Thus, if we retest initially negative specimens, we should get some that yield positive results on the repeat test. We did see some such results, but the sporadic nature of the positive results was impossible to interpret as anything but an occasional false-positive when we repeated the LCx and SDA assays. However, when we repeat tested initially negative AC2 specimens, a few (2/92) yielded the on/off pattern that we also observed with seeded specimens and with some initially positive specimens (Table 2). This suggests that there is still room for improving the sensitivity of NAATs. The variability could be due to minor technical variations due to the manual processing of specimens, irregular and less than optimal extraction of nucleic acid from the original specimen, or variability in the amplification (7, 13).
Our results lead to several conclusions. (i) Confirmatory testing is unnecessary; it complicates the handling of a positive NAAT result, as the initial failure to confirm a positive result is wrong more often than it is right, and adds cost to an already expensive screening test. (ii) Given that we can confirm >90% of positive NAAT results when calculated PPVs are <90% using specificity figures from the package inserts, the real NAAT specificity must be better than the levels given in the package inserts. (iii) There is still room for improving the sensitivity of NAATs, perhaps by better specimen preparation, automation, or target concentration.
|
ACKNOWLEDGMENTS |
|---|
This study was partly supported by funds from the CDC, Atlanta, Ga. (STD Program Grant H25/CCH904362-12-02).
Human subject approval for study 1 was by the University of California, San Francisco, Committee on Human Research. Human subject approval for study 2 was by the University of California, Berkeley, Committee on Human Research.
|
FOOTNOTES |
|---|
|
REFERENCES |
|---|
|
TopAbstractIntroductionMaterials and MethodsResultsDiscussionReferences |
|---|
This article has been cited by other articles:
| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Antimicrob. Agents Chemother. | Clin. Microbiol. Rev. |
|---|---|
| Clin. Vaccine Immunol. | ALL ASM JOURNALS |
|---|
