Coulter et al. are to be commended for their study evaluating the sensitivity of various tests for the diagnosis of Lyme disease (1). As the authors state, Lyme disease can be asymptomatic in its early stages and left untreated can develop chronic major manifestations. Studies evaluating available tests are quite important.
There are problems, however, with the data as presented. The combination of acute- and convalescent-phase serology with skin PCR is stated to have the highest sensitivity (100%), with serological testing with skin PCR almost as sensitive (92%). It would be easy for the reader miss the fact that these percentages were calculated using as a denominator the number of individuals who produced at least one positive test result from the set of tests being evaluated. These percentages would be useful if the intent of the study were to contribute to methodology for efficient selection of cases for future studies where high certainty of infection by Borrelia burgdorferi, the pathogen of Lyme disease, is required. The authors somewhat inappropriately expand the scope of their study when they comment the study supports the appropriateness of a published treatment guideline (3). Their definition of “sensitivity” becomes an issue.
The treating physician is likely to interpret test “sensitivity” in its broader sense: “How likely is a test to produce a positive result in an infected individual?” Using as their denominator the number of individuals in the study sample who produced at least one positive test result produces a result tangential to this question. The cohort producing at least one positive test result is quite different from the population the practitioner wants characterized: individuals infected with B. burgdorferi.
The dangers of misinterpretations of sensitivity statistics are obvious. Overstating sensitivities based on agreement with other tests, rather than characterization using known or highly probable infected subjects, encourages overconfidence for the treating physician in the tests being evaluated. Although the study attempts to present data from the cohort determined to be probable cases based on symptoms, there seems to be confusion in data presentation. For example, although 25 subjects were classified as probable for Lyme disease, the data show 32 subjects in this group positive on their initial serology. There are several other problems of this nature. Test results from subjects prospectively deemed unlikely to have infection were apparently not reported.
The authors are encouraged to clarify their data and emphasize their definition of sensitivity so that treating physicians do not misinterpret their findings and subsequently fail to diagnose patients with B. burgdorferi infection. Lack of prompt antimicrobial treatment for Lyme disease can result in a case of severe morbidity highly resistant to treatment (2). The sensitivity of our available tests should not be overestimated.
- Copyright © 2006 American Society for Microbiology
REFERENCES
Author's Reply
I thank Dr. Spinhirne for his comments and recognition of the importance of objective, evidence-based evaluations of Lyme disease diagnostics (3). His comments seem to focus on “ambiguous” definitions for sensitivity that obscure precise infection detection. This is puzzling since the accepted definition for sensitivity is very simple: sensitivity = (number of test positives/number of true positives) × 100.
The implication is that the denominator used in our study imprecisely estimates a true-positive Lyme disease population. However, the definition of such a population is key, and there have been significant flaws in many investigations (1, 4; IGeneX, Inc. website [http://www.igenex.com/lymeset4.htm ]; accessed 29 January 2006). Everyone agrees that objective findings must be used for scientific studies. Thus, no definition of Lyme disease can rest solely upon history or physical findings owing to inherent nonspecificity. We applied carefully researched, validated laboratory tests scrutinized by peer review and reproduced by other laboratories (3, 5) to a population for which Lyme disease was considered possible or probable for the majority. Using this approach, most would agree that identification of the bacterium by culture or PCR or the demonstration of a clear serological reaction provides an objective identification of infection. Owing to the uncertainty of clinical assessment, it seems very reasonable that this cohort is the most objectively defined “true-positive” population.
It is also important to avoid confusing agreement with sensitivity. Table 1 in our study revealed 32 subjects for whom initial serologic results and clinical assessments agreed, including 8 subjects initially seropositive with probable Lyme disease and also 43 initially seronegative assessed as NOT “probable” for Lyme disease. The intent was not to determine sensitivity but to illustrate the poor agreement between clinical and laboratory assessments, supporting our approach and the recommendations of the Infectious Diseases Society of America that call for maximizing positive and negative predictive value by integrating established clinical and laboratory studies (5). Our data also underscore the absolute requirement for an objective “gold standard.” For example, when the “gold standard” cohort includes those for whom any laboratory test was reactive AND patients who were suspected to have Lyme disease, the diagnostic sensitivity of all tests and combinations is lower. With the ever-broadening clinical criteria used by some to define Lyme disease (2), the sensitivity of testing could approach zero, where no objective criterion would be helpful and a diagnosis of Lyme disease could be supported by any subjective finding deemed suitable. The unfortunate outcome is “shopping” for a laboratory result that conforms to the clinical impression but is more likely to be false positive among tests with limited specificity.
Inevitably, health care practitioners must understand algorithms for laboratory confirmation with known levels of confidence. We generated data to provide sensitivity information based strictly upon laboratory investigations to provide a framework for evidence-based laboratory utilization. It is unfortunate that highly sensitive laboratory diagnostics for all phases of Lyme disease have not yet been developed. However, it would be a disservice to evidence-based medicine to misclassify patients by broadening gold standards to those derived from questionably objective clinical manifestations, subjective “accumulated experience,” or well-intentioned but unsupported opinions.
