ABSTRACT
The choice of a suitable automated system for a diagnostic laboratory depends on various factors. Comparative workflow studies provide quantifiable and objective metrics to determine hands-on time during specimen handling and processing, reagent preparation, return visits and maintenance, and test turnaround time and throughput. Using objective time study techniques, workflow characteristics for processing 96 and 192 tests were determined on m2000 RealTime (Abbott Molecular), Viper XTR (Becton Dickinson), cobas 4800 (Roche Molecular Diagnostics), Tigris (Hologic Gen-Probe), and Panther (Hologic Gen-Probe) platforms using second-generation assays for Chlamydia trachomatis and Neisseria gonorrhoeae. A combination of operational and maintenance steps requiring manual labor showed that Panther had the shortest overall hands-on times and Viper XTR the longest. Both Panther and Tigris showed greater efficiency whether 96 or 192 tests were processed. Viper XTR and Panther had the shortest times to results and m2000 RealTime the longest. Sample preparation and loading time was the shortest for Panther and longest for cobas 4800. Mandatory return visits were required only for m2000 RealTime and cobas 4800 when 96 tests were processed, and both required substantially more hands-on time than the other systems due to increased numbers of return visits when 192 tests were processed. These results show that there are substantial differences in the amount of labor required to operate each system. Assay performance, instrumentation, testing capacity, workflow, maintenance, and reagent costs should be considered in choosing a system.
INTRODUCTION
Automated instruments offer standardized processing technology for specimen extraction, specimen amplification, and detection of molecular targets (1). Minimal operator interaction is required, improving workflow, test throughput, and overall efficiency of laboratory operations. With clearance by the U.S. Food and Drug Administration and other regulatory agencies in Europe and elsewhere of several systems, automated molecular testing has become routine in clinical laboratory practice, ensuring diagnostic accuracy and improved result turnaround time (TAT). Studies have been published assessing clinical performances of molecular assays (2–8), and workflow and maintenance characteristics of automated instruments (9–16). In addition to performance and instrument-reagent costs, hands-on time required for testing and maintenance, in-process interaction, and time to results and test capacity are key metrics that should be considered because they can influence efficiency and labor costs. In this respect, workflow studies provide quantifiable and objective data to assist in choosing a system that is best suited for a given laboratory.
This study was conducted to determine the relative workflow and maintenance characteristics of five automated instruments commonly used for the diagnosis of sexually transmitted infections. For this purpose, the respective second-generation assays for C. trachomatis and N. gonorrhoeae were used to process 96 and 192 tests on each instrument. A related study (8) compared the diagnostic performance of these assays for C. trachomatis and N. gonorrhoeae on self-collected vaginal swabs and first-void urine samples on the respective automated instruments.
MATERIALS AND METHODS
Automated instruments and study sites.The study was carried out with the participation of four centers in Canada. RealTime m2000 CT/NG was performed on an m2000 RealTime instrument (Abbott Molecular, Des Plaines, IL) at Centre de santé et de services sociaux de Trois-Rivières, Trois-Rivières, Quebec, Canada. The ProbeTec ET CT/GC QX assay was carried out on a Viper XTR instrument (Becton Dickinson, Franklin Lakes, NJ) at Queen Elizabeth II Health Sciences Centre, Dalhousie University, Halifax, Nova Scotia, Canada. The cobas CT/NG 4800 test was performed on a cobas 4800 instrument (Roche Molecular Diagnostics, Pleasanton, CA) at Public Health Laboratory, St. John's, Newfoundland and Labrador, Canada. The Aptima Combo 2 (AC2) assay was performed on Tigris and Panther instruments (Hologic Gen-Probe, San Diego, CA) at St. Joseph's Healthcare, McMaster University, Hamilton, Ontario, Canada. Each site routinely used their respective systems for C. trachomatis and N. gonorrhoeae diagnosis. Descriptions of the five automated systems are shown in Table 1.
Description of automated platforms studied
Study description.Workflow and maintenance characteristics of each automated platform were determined based on 96 and 192 tests, including controls, per the package insert instructions for each system. By using a standard workflow template, the various steps for processing specimens through each assay were stratified into seven stages: (i) preanalytical interaction (instrument start-up, system fluid preparation and loading, consumable loading); (ii) reagent preparation and loading; (iii) sample preparation and loading; (iv) in-process interaction (number of times operator is required to return to the instrument during operation); (v) postanalytical interaction (unloading samples, reagents, and consumables); (vi) maintenance (cleanup and decontamination of instrument, sample and reagent trays, other accessories, and work surfaces, etc.); and (vii) automation (time each instrument is operational without operator interaction). Hands-on and automation times were measured precisely by recording start and end times. Based on the data obtained for the seven stages for processing 96 and 192 tests, the following four study parameters were determined for each instrument: (i) total hands-on time (total time required for manual interaction from the beginning to the end of an assay run, including daily maintenance); (ii) in-process interaction (number of times operator was required to return to the instrument during operation and total hands-on time required for return visits); (iii) time to results (time from startup to first and final results); and (iv) cumulative hands-on time for daily, weekly, and monthly maintenance, which was determined based on 96 tests per day, 5 testing days per week, and 20 testing days per month, respectively.
Normalization.As the study was based on 96 or 192 tests, processing and operator engagement times were normalized for instruments that are designed to process a greater number of tests. For example, preanalytical waste management in the Tigris instrument took 7 min 12 s; since this is performed for every 1,000 tests, the normalized time for 96 tests was calculated to be 41.5 s [(7 min 12 s/1,000) × 96]. Maintenance was performed and timed on individual instruments according to each manufacturer's recommended schedules regardless of the number of tests run and not normalized as part of this study.
Study procedure.All study sites processed vaginal swabs and urine specimens for C. trachomatis and N. gonorrhoeae in accordance with each manufacturer's instructions per standard operating procedures. Two investigators, working with an experienced technologist at each site, carried out the first study assessment over a 2-day period for 96 tests. The investigators initially reviewed the various steps of assay processing with the resident technologist and monitored a 96-test run of the assay to familiarize themselves with the sequence of assay processing carried out at each site. A month later, the investigators returned to each site and performed another evaluation for 192 tests.
For the 96-test study, all systems were assessed with a run made up of vaginal swabs and urine samples from female subjects and the required number of controls. For the 192-test study, two batches of 96 tests were assessed consecutively in m2000, cobas 4800, and Viper XTR (in throughput mode), as these instruments are batch-based systems with a maximum capacity of 96 tests. For Tigris, which is also a batch system but with greater batch capacity, 178 specimens with two controls were initially loaded to reach full capacity, with the additional 10 specimens and 2 controls loaded subsequently during operation. The Tigris carousel holds 9 racks, each having a capacity for 20 samples. Since Panther is a nonbatch, continuous-flow system, 118 specimens with two controls were initially loaded to full capacity, as there are 8 slots for racks holding 15 samples each, with the remaining 72 specimens loaded subsequently during operation. Test runs were timed precisely for each processing step, including daily maintenance using a workflow template. The results were tabulated for each stage of assay processing and summarized by the study parameters described above.
RESULTS
Table 2 summarizes hands-on time for each stage of assay processing and automation times for the five instruments for 96 tests. For the preanalytical interaction stage, Panther had the least hands-on time and Viper XTR had the most. For reagent preparation and loading, both Panther and Tigris showed the least hands-on time and Viper XTR showed the most. For sample preparation and loading, Panther had the least hands-on time and cobas 4800 had the most. In-process interactions for a 96-test run were minimal and were required only for m2000 and cobas 4800. For postanalytical interaction, Panther and Tigris had the least hands-on time compared with the other three instruments. There were substantial differences in daily maintenance hands-on time for the five systems, with Panther requiring the least and Viper XTR needing considerably more hands-on time than the rest. Overall, Panther showed the least total hands-on time, followed by Tigris, and Viper XTR had the most. In terms of automation, Viper XTR had the shortest automation time and m2000 the longest.
Hands-on and automation times for processing 96 testsa on five automated instruments
Figure 1 shows the total hands-on time for 96 and 192 tests for the five instruments. Viper XTR showed the most hands-on time and Panther the least hands-on time for processing both 96 and 192 tests. When doubling the number of tests from 96 to 192, both Tigris and Panther required the least amount of incremental hands-on time compared with the other three instruments.
Total hands-on time for each instrument for 96 and 192 tests. *, the 96-test workflow used walk-away mode, and the 192-test workflow used high-throughput mode. △, additional hands-on time was required for processing 192 samples relative to 96 samples.
In-process interaction return visits were mandatory for m2000 and cobas 4800 when 96 tests were processed. The m2000 required two return visits to load master mix and to transfer the amplification plate from m2000sp to m2000rt (hands-on time, 2 min). The cobas 4800 required one return visit to transfer the amplification plate from x480 to z480 (hands-on time, 2 min 25 s). These visits were critical and time sensitive. When 192 tests were processed, the return visits increased to four on m2000, requiring 22 min 22 s in hands-on time, and two visits for cobas 4800 required 14 min 58 s in hands-on time. Performing 192 tests required two return visits on Viper XTR (6 min 40 s hands-on time) to load additional specimens and consumables, and a single return visit on Panther (1 min 29 s) and Tigris (2 min 23 s) to load additional specimens.
Times to results are shown in Fig. 2. Results were batched, with the exception of those done on Panther. Viper XTR showed the shortest time to results, whether 96 (3 h 31 min) or 192 (5 h 8 min) tests were processed. In contrast, m2000 yielded the longest time to results (6 h 11 min for 96 tests and 10 h for 192 tests). The first result appeared on the Panther at 3 h 51 min. Results on the Panther were completed at 5 h 27 min when 96 tests were processed and at 7 h when 192 tests were processed.
Times to results for 96 and 192 tests. *, the 96-test workflow used walk-away mode, and the 192-test workflow used high-throughput mode. For Panther, the time to the first result was 3 h 51 min 2 s, the time to the last result for 96 samples was 5 h 27 min 2 s, and the time to the last result for 192 samples was 7 h 41 s.
All instruments required some daily maintenance, and some required weekly and monthly maintenance. The cumulative hands-on time for these activities are shown in Fig. 3. Based on 96 tests per day, we calculated that Panther required the least hands-on time for daily maintenance, at 3 min 26 s per testing day, compared to Viper XTR, which required 66 min per testing day (Table 2). Based on 5 testing days per week and 20 testing days per month, the cumulative differences for daily, weekly, and monthly maintenance would be substantial for Viper XTR at 22 h 30 min (Fig. 3). The m2000 and cobas 4800 required more hands-on time for weekly maintenance compared with the other 3 instruments but did not require monthly maintenance. There were minimal increases in maintenance hands-on time when 192 tests were performed, because daily hands-on time accounted for the bulk of time, and this remained unchanged across systems whether 96 or 192 tests were processed per testing day (data not shown).
Cumulative hands-on time for maintenance based on 96 tests per day, 20 working days per month.
DISCUSSION
In this study, we used objective time-motion criteria and a standard workflow template for consistency and accuracy. The assay processing stage-based model and the composite tabulation provided detailed data for accurate analysis and ensured quantifiable results.
The Abbott m2000 two-unit batch system allowed a maximum of 93 specimens with 3 controls to be processed in a run and required two return visits per run. When a batch of 96 tests was run, results were available 6 h 11 min from the start, confirming two previous reports of 6 h 20 min (7, 16). The total hands-on time, including daily maintenance for a 96-test run, was approximately 1 h, which was longer than the 43 min reported by Williams et al. (16). A second run of 96 tests doubled the total hands-on time and increased the mandatory visits to four, with results from the second run extended into the next work shift (Fig. 1 and 2). Similar observations of hands-on time and lesser throughput of the m2000 were reported previously (11, 15).
The BD Viper XTR single-unit batch system allowed a maximum of 92 specimens with 4 controls to be processed in a run, without mandatory return visits. Batched results for a 96-test run were available 3 h 31 min from the start, which is 1 h less than reported previously (16). Our study indicated the total hands-on time, including daily maintenance was 1 h 41 min for 96 tests. A previous study by Felder et al. (13) reported a total hands-on time of 35 min per run of 96 tests but did not include hands-on time for daily maintenance in the calculation. Daily maintenance for a 96-test run was considerably higher at 1 h 6 min than that of the other systems, and this also differs from the 41 min previously reported (16) (21 min for daily maintenance start plus 20 min postdetection shutdown maintenance at the finish). Based on 20 testing days per month of 96 tests per day, the maintenance hands-on time was the highest of all systems, at 22.5 h for the month. The BD Viper XTR was the only system that required preanalytical sample heating (15 min) and cooling (15 min) prior to loading of the samples onto the instrument. Although we did not perform the measurements for a third run of 96 tests, it appears that it could be completed without a substantial increase in daily maintenance and within an 8-h shift.
The Roche cobas 4800 two-unit batch system allowed testing of a maximum of 94 specimens with 2 controls in a run and required one mandatory return visit per run. With this instrument, batched results were available in 4 h 23 min from the start (Fig. 2), which was greater than the 3 h 57 min reported by Williams et al. (16) and the 3 h 30 min reported by Rockett et al. (17); both these studies did not show details of the measured time. The instrument we used in the study was not interfaced with the laboratory information system and thus required additional hands-on time for the sample preparation stage. This system also required more hands-on time for specimen preparation and loading due to uncapping of each sample tube prior to loading onto the instrument. Some laboratories may also adhere to the practice of recapping the tubes after processing, either for storage or prior to disposal, and this required additional hands-on time, which we have computed in this study. Total hands-on time, including daily maintenance, was 40 min for 96 tests, which was similar to the 36.5 min reported by Williams et al. (16), even though the hands-on-time required to recap sample tubes was specifically excluded from their data set.
The Hologic Gen-Probe Tigris single-unit batch system was designed for batches between 1 and 246 specimens. When 192 tests were processed, a higher capacity for specimens and consumables with larger reagent packs allowed one run of 180 tests to full capacity, followed by 12 more tests. This instrument required considerably less total hands-on time than the m2000, Viper XTR, and cobas 4800. Although Tigris is also a batch system, its design enables larger batching without substantial increases in hands-on time. Our observations are consistent with a previous report (11) that the Tigris required minimal hands-on time for the throughput achieved. The Tigris instrument was recently compared to the same three instruments with batches of 96 tests and was deemed easiest to operate (16). That comparison failed to record times on the Tigris for the same number of specimens to enable a proper comparison (109 and 132 tests on Tigris versus 96 tests on the other three instruments). That study indicated 77 min of hands-on time, including daily maintenance, for 132 tests, whereas our calculation was 28 min for 96 tests and 34 min for 192 tests.
The Hologic Gen-Probe Panther single-unit continuous-feed, random-access system allowed continuous loading after the initial 118 specimens were loaded to full capacity. The two run controls remain valid for 24 h, so more specimens could be loaded at any time without running additional controls or reagents, up to 250 tests. Panther showed the least total hands-on time of the five instruments evaluated. This is attributable mainly to fewer and simpler processing steps and the higher capacity for consumables, reagent pack size, and specimen loading. The design of this instrument allowed continuous access to reagents and samples, with loading and unloading in any order, while the instrument was processing. These characteristics may enable a more efficient workflow than can be achieved by instruments which are designed for batching.
Short TAT is considered an important function for treatable infections and would be considered a favorable attribute of any automated system in a clinical setting. However, for the treatment of chlamydial and gonorrhoeal infections, this may not be a critical factor, as the instruments included in this study all required more TAT than would be ideal for holding patients at point of care for treatment.
Automation times ranged between 3 and 5 h for 96 tests among different instruments, and laboratories may have different preferences for the length of uninterrupted automation time depending upon other manual tasks the operator may need to perform throughout the day. Another aspect that may matter in a clinical laboratory is the amount of space required for an instrument. Our study did not formally evaluate operational space requirements. A weakness of our study was the inability to measure system failures. While there were no assay run failures with any of the systems during the study, this could not be determined, as the study dealt with only a few assay runs on each system. We also did not study the relative requirements of plastic consumables in different systems, which could increase costs and have an environmental impact and may be a consideration in decision making.
In conclusion, considerable time requirement differences in various steps of assay processing are associated with different systems and are related in part to differences in the attendant hands-on time requirement and complexity of each instrument. Our study provides objective and quantifiable data on the relative workflow and routine maintenance characteristics of the five instruments studied. We measured considerable differences in terms of test capacity, hands-on time, in-process interaction, time to results, and maintenance hands-on time which may impact overall operational efficiency, workflow, and labor costs. When choosing a system, laboratories should consider the following: assay performance, instrumentation, reagent costs, testing capacity, workflow, and maintenance.
ACKNOWLEDGMENTS
This study was supported by a research grant from Hologic Gen-Probe Inc., San Diego, CA. Hologic Gen-Probe had no input in the study design, choice of instruments studied, data analysis, or preparation of the report.
We thank the following for meticulously reviewing assay processing steps with investigators, running assays, and support and cooperation throughout the study: Christine Peterson, Caroline Marchand, and Jocelyn Blouin, Centre de santé et de services sociaux de Trois-Rivières, Trois-Rivières, Quebec, Canada; Elizabeth Oates and Sandy Tucker, Public Health Laboratory, St. John's, Newfoundland and Labrador, Canada; and Catherine Roberts, Gale Naugle, and Joan MacDonald, Queen Elizabeth II Health Sciences Centre, Halifax, Nova Scotia, Canada. We also thank Laura Gilbert and Sandra Pike, Public Health Laboratory, St. John's, Newfoundland and Labrador, Canada, for assistance with the preparation of the manuscript.
S.R. has received research funding and consulted for Hologic Gen-Probe and received speaker honoraria. M.C. has received research funding and speaker honoraria from Hologic Gen-Probe. He has served as a consultant for Abbott Molecular and received research funding.
FOOTNOTES
- Received 30 December 2013.
- Returned for modification 31 January 2014.
- Accepted 10 April 2014.
- Accepted manuscript posted online 16 April 2014.
- Copyright © 2014, American Society for Microbiology. All Rights Reserved.
The authors have paid a fee to allow immediate free access to this article.