ABSTRACT
Variations in biological behavior suggest that each carcinogenic human papillomavirus (HPV) type should be considered individually in etiologic studies. HPV genotyping assays might have clinical applications if they are approved for use by the FDA. A widely used genotyping assay is the Roche Linear Array HPV genotyping test (LA). We used LA to genotype the HPV isolates from cervical specimens from women with the full spectrum of cervical disease: cervical cancer, cervical intraepithelial neoplasia (CIN), and HPV infections. To explore the feasibility and value of the automated reading of the LA results, we custom-designed novel, optical imaging software that provides optical density measurements of LA bands. We compared unmagnified visual examination with the automated measurements. The two measurements were highly associated. By either method, the threshold between a negative and a positive result was fairly sharp, with a clear bimodal distribution. Visually, most positive results were judged to be strong or medium, with fewer equivocal results categorized as weak (9.5% of positive samples), very weak (6.5% of positive samples), or extremely weak (7.7% of positive samples). The automated measurements of the intensities were significantly associated with the strength of the visual categories (P < 0.001). At the extremes of the automated signal intensities (≤20 units or ≥120 units), the bands were almost always categorized visually as negative and positive, respectively. In the equivocal zone (20 to 119 units), specimens were more increasingly likely to be judged to be visually positive as the number of other, definite infections on the same strip increased (P for trend < 0.001). Multiple, concurrent infections comprise ≥25% of HPV infections; thus, any systematic visual tendency that influences their evaluation when the result is equivocal should be minimized. Therefore, automated reading is probably worth development if easy-to-calibrate hardware and software can be optimized.
Cervical cancer and its immediate precursor, cervical intraepithelial neoplasia grade 3 (CIN3), are caused by persistent infection with 1 of approximately15 carcinogenic human papillomaviruses (HPVs) (7). However, the absolute and attributable risks that they will cause CIN3 and cancer vary among these carcinogenic HPV types (11); for example, HPV type 16 (HPV-16) is by far the most significant agent of cervical squamous cell carcinoma (4, 5), while HPV-18 is associated with a high risk of adenocarcinoma and squamous lesions that are difficult to detect in the precancerous state (12).
This variation in the biological behavior suggests that each carcinogenic HPV type be considered individually in etiologic studies and perhaps in clinical applications. Predicting the risk of progression is important for patient management, and HPV typing could contribute to this assessment (11). At the extreme, if HPV-16 persists, the risk of a diagnosis of CIN3 is as high as 30% within the first 3 to 5 years of infection (9). Therefore, refining our ability to detect and assess individual types is an important step in translational research.
Demonstrable robust performance is critical for any HPV genotyping technology prior to implementation in clinical practice, particularly when the intended use is to define viral persistence. The definition of persistence requires the consistent categorization of the viral type across multiple measurements, possibly in different laboratories, in order to make decisions about the need for immediate treatment or continued monitoring of patients.
A major issue in HPV typing is frequent coinfection with multiple HPV genotypes due to a common route of transmission, which occurs in up to 25% of infections (3, 6). Although the HPV types in patients coinfected with multiple HPV types have little effect on each other in terms of natural history (8), the presence of multiple genotypes with independent natural histories can make the clinical interpretation of such data quite complex. For example, one of two originally detected genotypes might disappear between annual visits, while the other type might persist and a third type might appear. Is only the persistent type important? It will likely take some time to develop functional clinical algorithms that effectively incorporate such complex data, even if HPV genotyping is nearly perfect. Substantial errors in analytic measurements would make the task hopeless, especially if genotyping artifacts are introduced due to intergenotype interferences in the context of infections with multiple HPV types.
Partly because of the analytical complexities of targeting multiple HPV types in a single specimen, no FDA-approved assay for HPV genotyping is currently available for clinical purposes. However, HPV genotyping is extensively used for research purposes, and various laboratory-developed protocols have been designed that use PCR amplification with general primer sets [primers GP5+ and GP6+, primers (PG)MY09 and (PG)MY11, and primer SPF10), which generally target the L1 gene of HPV, coupled with a detection system such as enzyme immunoassay, reverse line blot, bead array, or direct sequencing. Recently, commercial assays based on these protocols have been developed, including reverse blot assays, such as the Roche Diagnostics (Indianapolis, IN) Linear Array HPV genotyping test (LA; which is based on the PGMY primers) and the Innogenetics SPF10-InnoLiPA (line probe assay) HPV genotyping test (which is based on the SPF10 primers); bead array assays, such as the Multimetrix GmbH multiplex HPV genotyping kit assay (which is based on modified GP5+ and GP6+ primers); and silicon-based microarray assays, such as the Greiner PapilloCheck assay (which targets HPV E1).
LA (Roche Diagnostics) is one of the most widely used HPV genotyping assays (1) and has been granted approval for clinical use in some countries. While extensive scientific data have been generated by use of this assay, we have previously expressed concern about the analytical reproducibility of the method, particularly when multiple genotypes are present at low levels (2). One important factor potentially affecting the interpretation of data obtained by LA is the subjectivity of evaluating the hybridization signals on the strips; reading of the results is performed by naked-eye evaluation to determine the presence (positive result) or absence (negative result) of a continuous band in the predefined region of the strip.
We are using the LA with cytology specimens from the Study to Understand Cervical Cancer Early Endpoints and Determinants (SUCCEED) being conducted by the National Cancer Institute (NCI) and the University of Oklahoma. At present, we have genotyped HPV isolates from nearly 1,745 cases with the full spectrum of cervical disease: cervical cancer, CIN, and HPV infections. In order to address the subjectivity of strip analysis, we custom-designed novel, optical imaging software that yields measurements of the optical densities of the bands on LA strips. The generation of imaging data has permitted us to evaluate closely the influence of observer subjectivity, to assess the correlation between the visual evaluation and the signal strength in samples from women infected with single and multiple HPV types, and broadly, to explore the feasibility and value of automated reading of LA results.
MATERIALS AND METHODS
Study population.This study was based on specimens collected from the NCI-sponsored study SUCCEED conducted at the University of Oklahoma. Study procedures were independently approved by the University of Oklahoma Health Sciences Center (OUHSC) and NCI institutional review boards. Participants provided written informed consent prior to enrollment into the study. The study enrolled women referred for colposcopy to the Dysplasia Clinic based at OUHSC following an abnormal Pap smear result. Women attending the clinic for vaginal colposcopies, women under 18 years of age, women who had previously received chemotherapy or radiation as treatment for any cancer, and women pregnant at the time of their visit were excluded from the study. In total, 508 women with <CIN1, 461 women with CIN1, 311 women with CIN2, 288 women with CIN3, and 98 women with cervical cancer were enrolled.
Specimen collection.A physician conducted the pelvic and colposcopic examination according to local guidelines. Prior to the application of acetic acid, cervical cell samples were obtained with a Papette broom (Wallach Surgical, Orange, CT) and rinsed directly into a PreservCyt vial (Cytyc Corporation, Boxborough, MA), as described previously (10). The cytology specimen was used to prepare a ThinPrep slide (Cytyc Corporation) and was used for HPV testing with LA (Roche Diagnostics). Cervical secretions were collected with an ophthalmic sponge. Acetic acid and Lugol's iodine were topically applied to the cervix to identify suspected CIN. Biopsy specimens obtained from any woman with colposcopically suspected CIN were placed in separate prelabeled vials containing 10% buffered formalin.
DNA isolation.Two 1-ml aliquots were removed from each 20-ml PreservCyt vial prior to cytologic analysis with a ThinPrep slide. One aliquot was processed for DNA isolation with a QIAamp DNA blood mini kit (Qiagen, Germantown, MD), as described previously (2), and the second aliquot was stored for future research. Briefly, the cervical cells in the PreservCyt vial were pelleted by centrifugation at 12,000 × g for 2 min, and the cells were washed briefly in 1× Hank's balanced salt solution (Gibco, Carlsbad, CA) and then subjected to DNA extraction (with a 20,000 × g relative centrifugal force, according to the recommendations of Qiagen). The isolated DNA was stored at −70°C until it was subject to amplification by LA.
HPV genotyping.LA was performed as described previously (2). The procedure was performed according to the recommendations of the manufacturer but with the variation that 10 μl of template DNA was amplified and the amplified products were hybridized and detected with an automated Auto-LiPA staining system with 2.5 ml of each reagent per strip (compared to the 4.0 ml used for manual processing). In previous work, we observed that this variation in the procedure permits the detection of additional HPV types and stronger signals than use of the standard 50 μl of template DNA isolated from 250 μl from the PreservCyt vial by the protocol with QIAamp MinElute medium (2). The PCR-based LA detects 37 HPV genotypes (types 6, 11, 16, 18, 26, 31, 33, 35, 39, 40, 42, 45, 51, 52, 53, 54, 56, 57, 58, 59, 61, 62, 64, 66, 67, 68, 69, 70, 71, 72, 73, 81, 82, 83, 84, IS39, and CP6108). Each strip contains two cells with different concentrations of β-globin that are used as internal controls for determination of the adequacy of amplifiable DNA. Standard positive and negative controls were provided by the manufacturer and were processed with each batch of specimens. Up to 84 specimens were amplified at one time by LA (inclusive of controls). Up to 30 specimens were processed at one time during amplicon hybridization to the linear array and detection of the hybridization signal.
Imaging of LA strips.A digitized record of the hybridization signals on the LA strips was generated with an AutoChemi imaging system (UVP BioImaging Systems, Upland, CA) shortly after hybridization and detection were completed on the Auto-LiPA staining system. The strips were photographed while they were still wet because wet strips have more definitive banding, especially for weak signals. The UVP imaging system has a Hamamatsu Photonics (Hamamatsu, Japan) model C8484-51-03G camera with a Computar TV zoom lens that provides high-resolution images with an effective count of 1,344 by 1,023 (horizontal by vertical) pixels. Pictures were obtained in a gray scale and stored by using a 12-bit TIFF format.
Visual evaluation of LA strips.The LA results were initially evaluated by unmagnified examination of the strips by two of the authors (R.A.A. and S.T.D.). An unambiguous, continuous band was judged to indicate that biotinylated amplicons had hybridized to complementary sequences of the probes bound to the strips and was considered a positive result (Fig. 1). A reference guide overlay provided by Roche was used to relate the location of the band(s) on the strip to the HPV genotype(s) present. The evaluator also subjectively graded the intensity of each band as strong, moderate, weak, very weak, or extremely weak.
LA strips with arrows showing strong, medium, weak, very weak (v.-weak), and extremely weak (e.-weak) visual reading results. The third column for each signal displays the background-corrected signal strength measured automatically with the optical software. na, not available.
Automated evaluation of LA strips.The digital images of the LA results were evaluated at the Communications Engineering Branch, National Library of Medicine, Bethesda, MD. The custom-written computer algorithm was used to process 84 digitized images (with 12 to 20 LA strips per image) for the purpose of automatically extracting the intensity of the signal values from each HPV “cell” in the image. Note that the term “cell” is used to denote the location of an HPV-specific probe on the strip. Because each image was observed to contain a “background” component which varied spatially within the image, the algorithm estimated the magnitude of the local background and yielded two values for each cell: the absolute signal value (the signal value uncorrected for the background) and the relative signal value (the absolute signal value minus the magnitude of the local background). The algorithm generated an output of these two quantities on a scale from 0 to 1,000, with 0 corresponding to white (the weakest response possible) and 1,000 corresponding to black (the strongest response possible).
The algorithm operates fully automatically. The process is logically divided into three parts: (i) gross segmentation, which finds the region of interest in the image (i.e., the large rectangle which contains all of the LA strips) and then finds the vertical boundaries of each strip and the boundaries of the first and the last cells on each strip; (ii) fine segmentation, which estimates the coordinates of the central point in every cell; and (iii) feature extraction, which computes the absolute and relative signal values in each cell. The algorithm was implemented in the MATLAB program (version 7.01) and was executed on a 3.2-GHz/2-GB RAM Dell Optiplex GX270 Windows XP computer. The current version (version 0.1) of this software is engineering quality and was developed to assess the feasibility of automated reading of HPV linear array images and to generate data for this study; at the present time, it is being improved and is not available for public access. It takes the automated system approximately 5 s to compute the absolute and the relative signal values for one 20-strip image (20 strips × 40 cells per strip = 800 cells) and an additional 22 s to write the outputs to worksheets in Excel program files. Times are “stopwatch” times, as seen by the user. We continue to develop the algorithm to refine its capability to compute the relative signal strength and to improve its segmentation robustness in cases of images (see the information on exclusions due to technical reasons in Results) with unusual contrast, rotation, or large “clutter” caused by manual markings.
Statistical analysis.We used standard descriptive statistics to present the data from the visual evaluation and automatic measurements. We compared the two kinds of data with regard to the detection of HPV genotypes using analysis of variance and logistic regression.
To define an “equivocal” zone of automatic signal values (for the investigation of the impact of infections with multiple HPV types on strip interpretation), we took the intersection of (i) the largest 10% of the automatic readings from visually negative cells and (ii) the smallest 1% of automatic readings from visually clearly positive cells (strong, moderate, and weak cells). This intersection included 3,237 cells with an automatic range of 20 to 119, many of which were visually categorized as very weak or extremely weak. Within this range, we determined, using logistic regression, whether the number of other, definite signals (≥120 units) on the same strip influenced the probability that an equivocal signal would be called positive rather than negative.
RESULTS
In SUCCEED, 1,745 cytology samples were genotyped by LA. In the current study, we have included the results for 1,018 samples for which complete data from both visual evaluation and automated, high-resolution digital image analysis were available. Most cases excluded from analysis had incomplete imaging data, generally due to problems with photographic images, such as nonparallel LA strips or other anomalies. We continue to adapt the software in an attempt to accommodate these problematic images.
Images of the LA strips were given masked identifiers and were randomly analyzed. However, a slight difference between the specimens that were included and excluded (more cancers and fewer low-grade lesions in the latter) was marginally statistically significant (chi-square, 11.66; 5 degrees of freedom; P = 0.04) (Table 1) but was likely random. There was no significant relationship between the excluded batches and time that suggested the existence of a learning curve.
Comparison of final diagnoses in SUCCEED population and sample selected for present evaluationa
Figure 2 shows the aggregate distribution of the visual readings of the 1,018 HPV strips (37,666 cells), indicating that most of the cells did not show positive results. Among the 2,489 positive cells, strong signals were the most frequent result (55.8% of the positive cells), followed by moderate results (20.4% of the positive cells). In other words, the threshold between negative and positive was generally considered definite, with few equivocal results categorized as weak (9.5% of positive cells), very weak (6.5% of positive cells), or extremely weak (7.7% of positive cells).
Distribution of visual measurements of signal intensities of 37,666 cells of 1,018 LA strips from the SUCCEED study. The y axis was truncated to 1,800 counts. neg, negative; e, extremely weak; v, very weak; w, weak; m, moderate; s, strong.
Figure 3 shows the results of the automatic measurement of the background-corrected signal intensities of all the cells of the LA strips. Of note, the coefficient of variation of readings for the “globin high” cell showed a coefficient of variation of 12%. The graph has a bimodal distribution with a very large peak between 0 and 100 units, which largely corresponds to the negative visual results. A second peak is evident between 400 and 600 units, where the moderate and strong visual results are located. Between these two peaks is where we found most of the weak, very weak, and extremely weak signal strengths.
Distribution of background-corrected signals for 37,666 cells of 1,018 LA strips from the SUCCEED study. The y axis was truncated to 100 counts.
Figure 4 directly correlates the visual and automated measurements of signal strength. The interquartile ranges of the measured intensities were statistically significantly different between all visual categories and between the positive categories and the negative category, with a clear positive trend. Nonetheless, there was substantial overlap, especially for the negative, extremely weak, and very weak visual signals.
Comparison of the visual and automated measurements (background corrected signals) of signal intensity. Circles, outliers; asterisks, extreme values; 0, negative; 1, extremely weak; 2, very weak; 3, weak; 4, moderate; 5, strong.
Because of earlier work that suggested that infections with multiple HPV types were particularly difficult to analyze by LA, we explored whether having single or multiple positive signals on a strip influenced the likelihood for an equivocal signal to be visually categorized as positive or negative. The automatic measurement was taken as an objective reference standard for this analysis. We found that for automatically measured signal strengths between 20 and 119 units, there was a significant trend: equivocal signals were increasingly likely to be called positive by the naked eye as the number of other definite signals on the same strip increased (Table 2). In Fig. 1, a mixed probe signal with an intensity of 44 was called extremely weak on a strip with several signals, while on the adjacent strip with only one signal, an additional signal for HPV-16 with an intensity of 51, was not detected visually.
Effect of additional signals on strip of >119 light units on visual classification of equivocal signals (>20 and ≤119 light units)a
In our detailed comparison of the visual and automated methods for the evaluation of LA strips, the optical software found nine visual readings that were clearly misclassified. These nine misclassified signals occurred on four LA strips. In one case, an erroneous genotype was assigned to a signal that corresponded to the location of an adjacent HPV probe on the strip. Two strips were affected when a signal in one strip was attributed to the neighboring strip. In the fourth case, the genotype pattern reported visually was totally different from the automated results, suggesting a coding error.
DISCUSSION
The main goals of this study were to explore the feasibility of developing software for performing automatic visual reading of the results obtained with LA strips and to evaluate the advantages of performing that automatic evaluation compared with the performance of the reading by naked eye by an evaluator. While for other HPV genotyping platforms (chip based or bead based), automated evaluation systems are part of the test system, strip-based assays are so far dependent on visual evaluation.
Our results show a clear bimodal distribution of the signal strengths obtained by either visual or automatic evaluation of the LA strips. The two types of measurement were highly associated, providing assurance of the accuracy of the visual classification being used. There was a clear distinction between the large numbers of clearly negative signals and clearly positive signals (called moderate and strong).
Between the two peaks of this bimodal distribution of automated measurements, there was a zone that contained samples with equivocal signals. In this zone, the visual reading was much more likely to be positive for a given automatically measured signal intensity when at least one other HPV type was present in the specimen. Perhaps the presence of another positive band leads to the increased belief that a band considered equivocal by visual examination is truly positive. Because infections with multiple HPV types are very common, such a tendency of a reader could be an important source of error.
An equivocal zone is inherently unavoidable whenever a continuous measurement, such as signal intensity, which contains both biological and measurement variabilities is categorized as positive versus negative. An important finding of this study is that equivocal signals were more likely to be considered a positive result when there were multiple positive cells in the LA strip. This possible “optical bias” can be seen in Fig. 1. In the absence of other infections, an equivocal signal is more likely to be missed.
We recognize that the discordance due to test-specific differences observed when the results obtained with different primer/probe systems (InnoLiPA, Roche LA, etc.) are compared can be a major problem; however, when the same assay is used, it is important to eliminate all sources of avoidable variability, including the visual reading of the results. An accurate automated measurement of signal intensities on the LA strips would eliminate that variability. Additional research is still needed to determine the best cutoff point to be used to divide a positive result and a negative result before LA can be widely used for HPV typing and the assessment of viral persistence. We would need to distinguish situations when more sensitivity is and is not more important than more specificity. For example, for patients referred because of a positive screening test, the sensitivity of LA is probably of greater importance because of the need to track the course of an HPV-16 infection. A thorough evaluation would be needed for each use and each carcinogenic type. The use of the typing data for noncarcinogenic types detected by LA is of unclear clinical use. The possible exceptions may be HPV-6 and HPV-11, which cause condylomata.
While there were substantial intermethod differences in the reading of very weak signals, our study found only nine clear, nonsubtle misclassified signals from four different cases. This total could be considered a small number, because 1,018 samples were processed; however, in all four cases the mistake involved high-risk HPV types, showing that the results could have clinical implications for the women.
Although many image-processing programs are available, including some oriented toward the processing of densitometry images, we know of none like ours that is specifically customized for the efficient processing of HPV linear array strips. Any such software must address the problems of signal measurement and estimation of the background component for each cell on an image. Since the number of cells on an image is on the order of hundreds, some degree of automated processing by software that “knows” the structure of the image and that can recognize and adjust for image-to-image variations is essential for practical data collection. Although some of the required capabilities (e.g., the collection of absolute signal values at specified locations) are certainly available with off-the-shelf software, we know of no system that can be used without modification that has the comprehensive capability required.
A major factor arguing for the standardized, automatic evaluation of LA strips is that the test would be used to monitor women over time, and those sequential tests might be performed in different clinical laboratories. It would be important to minimize all factors that could vary between laboratories, including changing patterns of visual reading. Admittedly, the consistent calibration of automatic readers could emerge as a new issue if they are used. Our photographic protocol and prototype, open-source software led to many strips from random batches which could not be analyzed. Some of these problems can be overcome by additional programming. We note that our current effort was exploratory and was not intended to handle all the technical problems that can confound the reading of an LA strip. Eventually, LA reading hardware and software would need to read the correctly oriented strips themselves and not images.
Standardization of HPV typing is worth the effort because it forms the basis for HPV research and might play a central role because of its role in defining the persistence of HPV, in clinical decision making for patient management, and even in treatment. Any gain in precision and reliability is urged for the better and more adequate clinical management of the millions of women who are infected with HPV.
FOOTNOTES
- Received 29 January 2008.
- Returned for modification 24 March 2008.
- Accepted 2 June 2008.
- Copyright © 2008 American Society for Microbiology
