ABSTRACT
Multiple rapid antigen (Ag) tests for severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) have recently received emergency-use authorization (EUA) from the U.S. Food and Drug Administration (FDA). Although less sensitive than molecular detection methods, rapid antigen testing offers the potential for inexpensive, quick, decentralized testing. Robust analytical sensitivity data in comparison to reverse transcription-quantitative PCR (qRT-PCR) are currently lacking for many rapid antigen tests. Here, we evaluated the analytical sensitivity of the Abbott BinaxNOW COVID-19 Ag card using SARS-CoV-2-positive clinical specimens quantified by reverse transcription-droplet digital PCR (RT-ddPCR) and multiple FDA EUA qRT-PCR platforms using RNA standards. Initial and confirmatory limits of detection for the BinaxNOW COVID-19 Ag card were determined to be equivalent to 4.04 × 104 to 8.06 × 104 copies/swab. We further confirmed this limit of detection with 72 additional clinical samples positive for SARS-CoV-2 in either phosphate-buffered saline or viral transport medium. One hundred percent of samples with viral loads of >40,000 copies/swab were detected by rapid antigen testing. These data indicate that the BinaxNOW COVID-19 Ag card has an analytical sensitivity approximately equivalent to a generic qRT-PCR cycle threshold (CT) value of 29 to 30.
INTRODUCTION
Severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) continues to spread in the United States and across the world, causing tens of millions of cases and more than 1 million deaths. Testing for SARS-CoV-2 has predominantly been performed on molecular platforms with exquisite analytical sensitivities that can detect fewer than 100 viral RNA copies per ml of viral transport medium (VTM) (1). These analytical sensitivities result in the detection of low-level shedding in patients for weeks to months after initial infection, often with very low viral loads (2–5). A double-edged sword of the sensitivity offered by real-time reverse transcription-quantitative PCR (qRT-PCR) is the possibility that the molecular methodology detects RNA copies of virus that may not necessarily correlate with an active infection, especially if the result is a low viral load (6–8). However, assay sensitivity is critical for detecting additional cases where low viral loads in the nares are present during active lower respiratory tract infection (9).
Despite the current provision of more than 1 million SARS-CoV-2 tests per day in the United States, molecular testing may not be able to scale much further. Since the beginning of the pandemic, many have looked to antigen (Ag) tests to provide rapid, inexpensive, and decentralized testing that might potentially reduce transmission with the thought that antigen tests could successfully detect infectious cases (10, 11). However, the demand for antigen testing has outstripped the associated data on its performance, most notably with the $750 million outlay by the U.S. government for the Abbott BinaxNOW COVID-19 Ag card based on data from just over 100 specimens in the emergency-use authorization (EUA) application (12–14). Only six samples with a cycle threshold (CT) value of >33 were tested as part of the submission for U.S. Food and Drug Administration (FDA) authorization for the BinaxNOW COVID-19 Ag card. Other antigen tests have reported specificities of 100% based on testing tens of specimens, but real-world performance has not supported these estimates (15–17).
Accurately understanding the trade-off of sensitivity and speed is critical to achieving the right balance of diagnostics used in different settings. While determination of clinical sensitivity in vivo is the gold standard, such studies can be complicated by the ordering of swabs and by differences in patient anatomy, collector experience, and transport (18–20). Here, we evaluated the analytical sensitivity of the Abbott BinaxNOW COVID-19 Ag card in a central laboratory by subjecting the lateral flow assay to known amounts of SARS-CoV-2 quantified by a variety of qRT-PCR platforms.
MATERIALS AND METHODS
Specimen collection and quantification.This work was approved by the University of Washington institutional review board (IRB) (STUDY00010205). Deidentified samples used for the limit of detection (LoD) experiments were collected from patient specimens sent to the University of Washington Medical Center (UWMC) in Seattle, WA, for clinical diagnostic testing. Two nasopharyngeal swabs (NPSs) transported in phosphate-buffered saline (PBS) with high-viral-load CTs of 16.0 and 21.9 by the Panther Fusion SARS-CoV-2 assay (Hologic, Marlborough, MA, USA) were pooled (21). This pool was quantified on the Roche cobas 6800 SARS-CoV-2 platform (Roche, Basel, Switzerland) using AccuPlex verification panel RNA standards (SeraCare Life Sciences, Milford, MA, USA) as well as by the reverse transcription-droplet digital PCR (RT-ddPCR) system (Bio-Rad, Hercules, CA, USA) using the CDC N1 primer (22, 23).
Four hundred microliters of input material was utilized for the Roche cobas SARS-CoV-2 assay, and 500 μl was used for detection using the Hologic Panther Fusion SARS-CoV-2 assay. For RT-ddPCR, nucleic acids were extracted from 200 μl of the sample using the QIAamp viral RNA minikit (Qiagen, Hilden, Germany) and eluted into 100 μl distilled water (dH2O); 10 μl of this extracted RNA was used as the template in a 25-μl amplification reaction mixture, with samples run in triplicate and evaluated by mean quantitation (23). The cobas 6800 calibration standards reported CTs for the E gene of 27.72, 30.89, and 34.38 for 1 × 105, 1 × 104, and 1 × 103 calibration standards, respectively. The Panther Fusion assay standards reported CTs for the open reading frame 1ab (ORF1ab) gene target of 27.3, 30.8, and 34.5 for 1 × 105, 1 × 104, and 1 × 103 calibrators, respectively. According to Hologic’s FDA package insert, the Panther Fusion SARS-CoV-2 assay 1× LoD averaged from 10 contrived swab specimens was a CT of 35.6, while the CT cutoff is approximately 42 cycles (https://www.hologic.com/sites/default/files/2020-09/AW-21159-001_004_01.pdf). Similarly, Roche’s FDA EUA reported the cobas 6800 LoD at mean CTs of 32.7 for ORF1ab and 35.2 for the E gene, with a dynamic CT cutoff of around 44 cycles, although it runs 50 cycles on the instrument (https://www.fda.gov/media/136049/download).
The Roche cobas assay, RT-ddPCR, and the Abbott BinaxNOW rapid antigen test were all performed in parallel. Additionally, BinaxNOW provided positive swabs that were tested for each shipment lot number to account for reagent and Ag assay quality control. For analytical negative controls, at both initial and confirmatory LoD dilutions, a clinical NPS previously determined to be negative by the Panther Fusion assay was subjected to the rapid Ag test.
Determination of analytical sensitivity.For the initial LoD, rapid Ag tests were run in triplicate with serial 10-fold PBS dilutions of the virus. First, a range of volumes (dH2O) from 25 to 100 μl was applied to each kit-provided sterile foam-tipped applicator to visually evaluate saturation; after soaking the swab in dH2O, any liquid remaining in the microcentrifuge tube was measured with a pipette. Without oversoaking or underexposing the swab, 50 μl was determined to be appropriate for saturation. Next, 50 μl of the quantified sample was aliquoted into microcentrifuge tubes, where each swab was inserted, gently rotated to absorb liquid, and subjected to the rapid antigen test according to the manufacturer’s protocol.
To account for liquid absorbed by the swab that might dilute the extraction buffer, a separate set of triplicate swabs was also allowed to air dry for 25 min after being saturated. To account for the potential impact of PBS on the antigen assay reaction, the original pool was also diluted in triplicate with dH2O for comparison. No difference was detected in either comparison for dH2O-diluted swab saturation and air-dried swabs. The initial LoD in triplicate was determined at the same dilution level regardless of whether the virus applied to the swab was diluted in PBS, diluted in water, or air dried (data not shown). All COVID-19 Ag cards were read after 15 min, and samples were never frozen and stored at 4°C.
RESULTS
The original positive sample pool yielded CTs of 17.26 for target 1 (ORF1ab) and 17.27 for target 2 (E gene) with the Roche cobas SARS-CoV-2 assay. Based on RNA standard quantification using the E gene primer set, this result corresponded to 1.13 × 108 copies/ml, while RT-ddPCR quantified the sample at 5.65 × 107 copies/ml based on the CDC N1 primer set. Initial LoDs of BinaxNOW were recognized as the last dilution set to detect 3/3 (100%) positive samples. At first, this put the detection range between 10-fold and 100-fold dilutions of the neat, positive pool (or 2.83 × 105 to 2.83 × 104 copies/swab when quantified by RT-ddPCR) (Table 1).
Initial and confirmatory LoDs for the Abbott BinaxNOW COVID-19 Ag cardb
Further dilution sets in triplicate narrowed this initial LoD range to 1:70 to 1:80 dilutions. Confirmatory LoDs were established with 20 replicates at each 1:70 and 1:80 dilution and characterized by ≥95% (19/20) positive replicates detected. Initial and confirmatory LoDs for Abbott’s BinaxNOW rapid antigen test were both determined at a 1:70 dilution of the original pooled positive samples, corresponding to 8.06 × 104 E gene copies/swab by RNA standards run on Roche cobas 6800 or 4.04 × 104 N gene copies/swab by RT-ddPCR (Table 1).
Based on the RNA standard curve on the cobas 6800 platform and a typical dilution of a swab into 3 ml VTM or PBS, these limits of detection correspond to E gene CTs of approximately 30.5 and 29.5, respectively. We then took the 50-μl volume at the LoD (1:70 dilution of the neat, positive pool), applied it to the kit-provided foam-tipped swab, and diluted it in 3 ml of PBS, similar to how a clinical NPS would be treated for molecular detection. The cobas assay quantified this sample with CTs of 28.68 (ORF1ab) and 29.08 (E gene).
After establishing the LoD range at 4.04 × 104 to 8.06 × 104 copies/swab, we next expanded the sample set to include 72 additional clinical positive specimens. These fresh NPSs were previously positive by Hologic Panther Fusion with ORF1ab CTs ranging from 14.6 to 39.9, corresponding to viral loads of 3.29 × 108 to 2.91 × 101 copies/ml, and stored in either PBS or VTM. For all specimens with >40,000 copies/swab (n = 24), the Abbott BinaxNOW-19 Ag card correctly detected 100% of deidentified samples (Table 2). Notably, multiple samples beyond the LoD were still detected, with some having as few as an estimated 2,400 copies/swab.
COVID-19 Ag card rapid test across a range of SARS-CoV-2-positive specimensa
DISCUSSION
Here, we describe a detailed examination of the analytical sensitivity of the BinaxNOW COVID-19 Ag card in a central laboratory. We estimate a limit of detection of 4.04 × 104 to 8.06 × 104 viral copies/swab, corresponding to a CT of approximately 30. These data generally agree with Abbott’s package insert, which separated samples based on a CT cutoff of 33. These data suggest a difference of approximately 6 to 7 CTs between the LoD of this antigen test and that of RT-PCR tests, indicating an ∼100-fold difference in sensitivity. Importantly, the difference in the CT is notable as several studies have been able to culture virus at viral loads above a CT of 30 but rarely above a CT of 34 (8, 24, 25). According to work by Singanayagam et al., viable viruses with CTs of >35 were found in 8.3% (5/60) of patient samples (26). Importantly, putting the limit of detection in terms of qRT-PCR CTs can vary by platform based on extraction and elution volumes, viral transcripts targeted by the assay, and the amount of transport medium that the swab is diluted in before it is amplified for molecular detection. To avoid being dependent on one assay for quantitation, we quantified our original pooled positive samples in PBS on multiple platforms using multiple quantitation methods in our clinical laboratory in terms of viral copies per swab. These values of copies per swab correlate the amount of virus that each rapid antigen test sees with molecular detection methods.
This study was chiefly limited by the testing of specimens sent in transport media to a central laboratory. Rapid antigen tests are not generally meant to be tested on specimens in transport media, overcoming their higher limits of detection by sampling a whole swab. We took measures to ensure that our contrived specimens would directly answer the question of limits of detection by first using specimens in PBS for the LoD determination since the compositions of viral transport media and other complex matrices have been shown to inhibit rapid antigen assays (14, 27). We focused on rigorously interrogating the analytical sensitivity of the assay and did not evaluate assay specificity as we presumed that specimens in transport media would not adequately reflect relevant interferences found in direct nasal specimens. It has been well established that molecular detection is not grossly affected by PBS versus VTM (28). Moreover, two studies have used PBS or VTM samples for antigen testing (13, 29). It is possible that additional studies could better inform the equivalency of VTM versus PBS. We also note that the assays performed here do not adjudicate the “infectiousness” of the individuals associated with these specimens or the public health impact of different testing algorithms, which can be measured only by detailed clinical investigation outside the laboratory.
Our work also did not measure the analytical specificity of the antigen test as we believed that we could not adequately model specificity under the contrived conditions in the laboratory. Understanding assay specificity is critical to estimating the number of false-positive results that may occur in a given testing environment. Of note, the FDA recently administered a press release on 3 November 2020, warning clinical laboratory staff and health care providers to be aware of the potential for false-positive test results with rapid antigen tests (30). Finally, in order to compare sensitivities in terms of qRT-PCR, we reported the LoD in viral RNA copies, even though viral RNA is not targeted by rapid antigen tests. Although CTs can vary between platforms, laboratories, analytical parameters, and even technicians and runs, LoD CT values are also reported here for semiquantitative reference.
Having access to affordable, rapid testing is critical to reducing transmission during a pandemic. Although rapid antigen tests can be convenient and inexpensive, here, we confirm that thousands to tens of thousands of viral copies are necessary for detection, which is significantly greater than that required for qRT-PCR (12, 13, 31). These limitations in sensitivity are balanced by the ease of use and short turnaround time of the antigen test (32, 33). Our work more closely approximates the inherent abilities and limitations of antigen cards to detect virus, beyond the myriad of preanalytical variables that may differ between tests. These estimates are critical for evaluating when to use antigen versus qRT-PCR testing and to more accurately model how many cases are detected and missed by antigen tests (34). A limit of detection at ∼40,000 copies/swab was also recently seen in outside work performed in parallel (35). More epidemiological and clinical research work, beyond viral culture, is required to determine whether the viral loads detected by antigen cards correspond to “infectious” virus and whether rapid diagnostics will ultimately help to detect and reduce viral transmission.
ACKNOWLEDGMENTS
We thank Aria Bovell, V. Michele Ladd, and Anna Chao for assistance with data acquisition. We thank Reed Schuler, Romesh Gautom, and the Washington State Department of Health for providing the kits.
A.L.G. reports research support from Abbott Laboratories, independent of this work. Abbott had no role in the design or execution of the study.
FOOTNOTES
- Received 12 November 2020.
- Returned for modification 7 December 2020.
- Accepted 10 December 2020.
- Accepted manuscript posted online 11 December 2020.
- Copyright © 2021 American Society for Microbiology.
This article is made available via the PMC Open Access Subset for unrestricted noncommercial re-use and secondary analysis in any form or by any means with acknowledgement of the original source. These permissions are granted for the duration of the World Health Organization (WHO) declaration of COVID-19 as a global pandemic.
