Skip to main content
  • ASM
    • Antimicrobial Agents and Chemotherapy
    • Applied and Environmental Microbiology
    • Clinical Microbiology Reviews
    • Clinical and Vaccine Immunology
    • EcoSal Plus
    • Eukaryotic Cell
    • Infection and Immunity
    • Journal of Bacteriology
    • Journal of Clinical Microbiology
    • Journal of Microbiology & Biology Education
    • Journal of Virology
    • mBio
    • Microbiology and Molecular Biology Reviews
    • Microbiology Resource Announcements
    • Microbiology Spectrum
    • Molecular and Cellular Biology
    • mSphere
    • mSystems
  • Log in
  • My alerts
  • My Cart

Main menu

  • Home
  • Articles
    • Current Issue
    • Accepted Manuscripts
    • COVID-19 Special Collection
    • Archive
    • Minireviews
  • For Authors
    • Submit a Manuscript
    • Scope
    • Editorial Policy
    • Submission, Review, & Publication Processes
    • Organization and Format
    • Errata, Author Corrections, Retractions
    • Illustrations and Tables
    • Nomenclature
    • Abbreviations and Conventions
    • Publication Fees
    • Ethics Resources and Policies
  • About the Journal
    • About JCM
    • Editor in Chief
    • Editorial Board
    • For Reviewers
    • For the Media
    • For Librarians
    • For Advertisers
    • Alerts
    • RSS
    • FAQ
  • Subscribe
    • Members
    • Institutions
  • ASM
    • Antimicrobial Agents and Chemotherapy
    • Applied and Environmental Microbiology
    • Clinical Microbiology Reviews
    • Clinical and Vaccine Immunology
    • EcoSal Plus
    • Eukaryotic Cell
    • Infection and Immunity
    • Journal of Bacteriology
    • Journal of Clinical Microbiology
    • Journal of Microbiology & Biology Education
    • Journal of Virology
    • mBio
    • Microbiology and Molecular Biology Reviews
    • Microbiology Resource Announcements
    • Microbiology Spectrum
    • Molecular and Cellular Biology
    • mSphere
    • mSystems

User menu

  • Log in
  • My alerts
  • My Cart

Search

  • Advanced search
Journal of Clinical Microbiology
publisher-logosite-logo

Advanced Search

  • Home
  • Articles
    • Current Issue
    • Accepted Manuscripts
    • COVID-19 Special Collection
    • Archive
    • Minireviews
  • For Authors
    • Submit a Manuscript
    • Scope
    • Editorial Policy
    • Submission, Review, & Publication Processes
    • Organization and Format
    • Errata, Author Corrections, Retractions
    • Illustrations and Tables
    • Nomenclature
    • Abbreviations and Conventions
    • Publication Fees
    • Ethics Resources and Policies
  • About the Journal
    • About JCM
    • Editor in Chief
    • Editorial Board
    • For Reviewers
    • For the Media
    • For Librarians
    • For Advertisers
    • Alerts
    • RSS
    • FAQ
  • Subscribe
    • Members
    • Institutions
Immunoassays

Multicenter Evaluation of the Clinical Performance and the Neutralizing Antibody Activity Prediction Properties of 10 High-Throughput Serological Assays Used in Clinical Laboratories

C. Therrien, B. Serhir, M. Bélanger-Collard, J. Skrzypczak, D. K. Shank, C. Renaud, J. Girouard, V. Loungnarath, M. Carrier, G. Brochu, F. Tourangeau, B. Gilfix, A. Piche, R. Bazin, R. Guérin, M. Lavoie, V. Martel-Laferrière, C. Fortin, A. Benoit, D. Marcoux, N. Gauthier, A. M. Laumaea, R. Gasser, A. Finzi, M. Roger
Yi-Wei Tang, Editor
C. Therrien
aLaboratoire de Santé Publique du Québec, Institut Nationale de Santé Publique, Sainte-Anne-de-Bellevue, Québec, Canada
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for C. Therrien
B. Serhir
aLaboratoire de Santé Publique du Québec, Institut Nationale de Santé Publique, Sainte-Anne-de-Bellevue, Québec, Canada
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for B. Serhir
M. Bélanger-Collard
aLaboratoire de Santé Publique du Québec, Institut Nationale de Santé Publique, Sainte-Anne-de-Bellevue, Québec, Canada
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
J. Skrzypczak
aLaboratoire de Santé Publique du Québec, Institut Nationale de Santé Publique, Sainte-Anne-de-Bellevue, Québec, Canada
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
D. K. Shank
aLaboratoire de Santé Publique du Québec, Institut Nationale de Santé Publique, Sainte-Anne-de-Bellevue, Québec, Canada
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
C. Renaud
bCentre Hospitalier Universitaire Sainte-Justine, Montreal, Québec, Canada
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
J. Girouard
cCentre Hospitalier Universitaire de Québec, Québec, Québec, Canada
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
V. Loungnarath
cCentre Hospitalier Universitaire de Québec, Québec, Québec, Canada
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
M. Carrier
dHôpital Cité de la Santé, Laval, Québec, Canada
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
G. Brochu
eCentre Hospitalier Affilié Universitaire Régional, Trois-Rivières, Québec, Canada
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
F. Tourangeau
fCentre Hospitalier Régional de Rimouski, Rimouski, Québec, Canada
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
B. Gilfix
gCentre Universitaire de Santé McGill, Montreal, Québec, Canada
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
A. Piche
hCentre Hospitalier Universitaire de Sherbrooke, Sherbrooke, Québec, Canada
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
R. Bazin
iHéma-Québec, Québec, Québec, Canada
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
R. Guérin
jHôpital de Chicoutimi, Chicoutimi, Québec, Canada
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
M. Lavoie
jHôpital de Chicoutimi, Chicoutimi, Québec, Canada
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
V. Martel-Laferrière
kDépartement de Microbiologie, Infectiologie, Immunologie, Université de Montréal, Montreal, Québec, Canada
lCentre de Recherche du Centre Hospitalier de l’Université de Montréal, Montreal, Québec, Canada
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
C. Fortin
kDépartement de Microbiologie, Infectiologie, Immunologie, Université de Montréal, Montreal, Québec, Canada
lCentre de Recherche du Centre Hospitalier de l’Université de Montréal, Montreal, Québec, Canada
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
A. Benoit
mCentre Hospitalier de l’Université de Montréal, Montreal, Québec, Canada
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
D. Marcoux
nHôpital de Lévis, Lévis, Québec, Canada
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
N. Gauthier
oHôpital Sacré-Cœur de Montréal, Montreal, Québec, Canada
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
A. M. Laumaea
kDépartement de Microbiologie, Infectiologie, Immunologie, Université de Montréal, Montreal, Québec, Canada
lCentre de Recherche du Centre Hospitalier de l’Université de Montréal, Montreal, Québec, Canada
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
R. Gasser
kDépartement de Microbiologie, Infectiologie, Immunologie, Université de Montréal, Montreal, Québec, Canada
lCentre de Recherche du Centre Hospitalier de l’Université de Montréal, Montreal, Québec, Canada
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
A. Finzi
kDépartement de Microbiologie, Infectiologie, Immunologie, Université de Montréal, Montreal, Québec, Canada
lCentre de Recherche du Centre Hospitalier de l’Université de Montréal, Montreal, Québec, Canada
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for A. Finzi
M. Roger
aLaboratoire de Santé Publique du Québec, Institut Nationale de Santé Publique, Sainte-Anne-de-Bellevue, Québec, Canada
kDépartement de Microbiologie, Infectiologie, Immunologie, Université de Montréal, Montreal, Québec, Canada
lCentre de Recherche du Centre Hospitalier de l’Université de Montréal, Montreal, Québec, Canada
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Yi-Wei Tang
Cepheid
Roles: Editor
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
DOI: 10.1128/JCM.02511-20
  • Article
  • Figures & Data
  • Info & Metrics
  • PDF
Loading

ABSTRACT

As the coronavirus disease 2019 (COVID-19) pandemic second wave is emerging, it is of the upmost importance to screen the population immunity in order to keep track of infected individuals. Consequently, immunoassays for severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) with high specificity and positive predictive values are needed to obtain an accurate epidemiological picture. As more data accumulate about the immune responses and the kinetics of neutralizing-antibody (nAb) production in SARS-CoV-2-infected individuals, new applications are forecast for serological assays such as nAb activity prediction in convalescent-phase plasma from recovered patients. This multicenter study, involving six hospital centers, determined the baseline clinical performances, reproducibility, and nAb level correlations of 10 commercially available immunoassays. In addition, three lateral-flow chromatography assays were evaluated, as these devices can be used in logistically challenged areas. All assays were evaluated using the same patient panels in duplicate, thus enabling accurate comparison of the tests. Seven immunoassays examined in this study were shown to have excellent specificity (98 to 100%) and good to excellent positive predictive values (82 to 100%) when used in a low (5%)-seroprevalence setting. We observed sensitivities as low as 74% and as high as 95% at ≥15 days after symptom onset. The determination of optimized cutoff values through receiver operating characteristic (ROC) curve analyses had a significant impact on the diagnostic resolution of several enzyme immunoassays by increasing the sensitivity significantly without a large trade-off in specificity. We found that spike-based immunoassays seem to be better correlates of nAb activity. Finally, the results reported here will add to the general knowledge of the interlaboratory reproducibility of clinical performance parameters of immunoassays and provide new evidence about nAb activity prediction.

INTRODUCTION

A novel variant of coronavirus termed severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) was promptly identified as the etiologic agent of a pneumonia outbreak in Wuhan, China, in December 2019 (1). This zoonotic virus spread rapidly among the immunologically naive human populations of the world in only a few months, creating the biggest pandemic crisis since the swine flu in 2009-2010 (2). SARS-CoV-2 is classified in the beta-coronavirus group upon phylogenetic analysis, sharing high genomic identity (96%) with the bat coronavirus RaTG13 and 80% nucleotide identity with the 2002-2003 pandemic SARS-CoV (3). Conversely, its genetic relatedness to other human beta-coronaviruses, such as OC-43 and HKU-1, and the human alpha-coronaviruses 229E and NL63 is significantly lower (4). The laboratory diagnosis of SARS-CoV2 infection is routinely made by the detection of viral RNA in nasopharyngeal samples in symptomatic patients by real-time reverse transcriptase PCR assays (5). The sensitivity of the molecular assays is good in the acute phase of the disease (6, 7), but they cease to be useful once the virus is cleared by the immune response of the patient, around 2 weeks following the onset of symptoms (8). The lack of sensitivity of the molecular assay as a function of time could impair the correct clinical assignment of patients, and this could contribute to the spread of the virus. In addition, some reports indicated that asymptomatic carriers who are not routinely tested could also contribute to spreading the virus (9).

Serological methods were proposed for use as screening tools for the epidemiological assessment of the immune profiles of the general population (10). Serological assays could also help to assess the immune status of convalescent-phase sera used to treat severely ill patients or for the validation of the effectiveness of vaccination trials. While serology is an indirect assay, it can provide useful insights about the exposition of asymptomatic individuals by the detection of specific antibodies. Thus, the challenge posed by this novel viral variant is the selection of highly specific antibody detection assays which could discriminate this virus from other related viruses and also be sensitive enough to be useful in the acute phase of the disease. Because of the necessity to screen immune profiles of several thousands of individuals in a short period of time, the validation of several assays available on high-throughput serology analyzers is a prerequisite to accomplish this task. In this study, we sought to evaluate the clinical performance and the virus neutralization activity prediction properties of 10 high-throughput serological enzyme immunoassays (EIA). We also evaluated three lateral flow rapid antibody detection devices, since these tests could be useful for screening purposes in distant settings.

MATERIALS AND METHODS

Study design.This multicenter evaluation study, initiated by the Québec provincial health authorities, was a collective effort led by the Laboratoire de Santé Publique du Québec (LSPQ). A total of 10 clinical laboratories were selected for this study, including the reference laboratory (LSPQ). These laboratories were grouped in separate testing sites according to their affiliations to university hospital centers, as follows: site 1, Centre Hospitalier de l’Université de Montréal (CHUM) and Centre Hospitalier Universitaire Sainte-Justine; site 2, Centre Hospitalier de l’Université Laval, Hôpital de l’Enfant-Jésus, and Hôpital du Saint-Sacrement; site 3, Hôpital Cité de la Santé; site 4, McGill University Health Center–Glen Site and St. Mary’s Hospital; site 5, Centre Hospitalier de l’Université de Sherbrooke; and site 6, LSPQ.

This study was conducted in accordance with the Declaration of Helsinki in terms of informed consent and approval by an appropriate institutional board and with the regulations concerning ethical reviews in Quebec provincial jurisdiction, particularly those specified in the Civil Code (http://legisquebec.gouv.qc.ca/fr/ShowDoc/cs/CCQ-1991) and in subsequent internal review board practice. Informed consent was obtained for all participating subjects, and the study was approved by Quebec Public health authorities.

Ten commercial antibody assays running on high-throughput immunoanalyzer platforms currently found in hospital laboratories and three lateral-flow chromatographic immunoassays (LFCIA) were selected for the study and tested at six designated sites. Each of the assays was tested in two separate laboratories to assess the reproducibility of the data except for the LFCIA which were tested only once at site 6. All of the selected assays had received the emergency usage authorization (EUA) by the Food and Drug Administration of the United States, while seven commercial enzyme immunoassays (Abbott IgG, Bio-Rad total antibody, DiaSorin IgG, Euroimmun IgG, Ortho-Clinical total antibody, Roche total antibody, and Siemens total antibody) had been authorized for use in clinical settings by Health Canada.

Patient cohorts.SARS CoV-2 samples (sensitivity panel; n = 176) were collected from patients with various clinical symptoms, including 156 sera and 20 plasma samples (Table 1). All 176 patients were confirmed positive for SARS CoV-2 infection by real-time reverse transcription-PCR (RT-PCR) on nasopharyngeal specimens; 170 of these patients were symptomatic and six were asymptomatic. The average age of the patients was 61 years. Fifty-six (32%) were male, 100 (57%) were female, and 20 (11%) were of unspecified gender. Forty-six samples were obtained at 1 to 7 days, 36 between 8 and 14 days, 60 between 15 and 34 days, and 34 more than 5 weeks after symptom onset. For asymptomatic patients as well as few symptomatic patients without a precise date of symptom manifestations, the time of onset was estimated as 48 h before the first positive SARS CoV-2 RT-PCR result. Among the patients with complete clinical information (n = 48), 32 (67%) were hospitalized, 29 (60%) required oxygen support, 8 (17%) were admitted to intensive care units, and 42 (88%) reported one or more symptoms (Table 1). The reported symptoms included one or more of the following: fever, myalgia, anosmia, ageusia, nasal congestion, pharyngitis, anorexia, and diarrhea.

View this table:
  • View inline
  • View popup
  • Download powerpoint
TABLE 1

Baseline characteristics of SARS CoV-2 patients (n = 176)

The first 67 SARS-CoV-2 samples collected for the sensitivity panel were selected for virus neutralization activity measurements. Twenty-one samples were obtained at 1 to 7 days, 14 between 8 and 14 days, 26 between 15 and 34 days, and 6 more than 5 weeks after symptom onset.

Specificity and cross-reactivity of the serological assays were evaluated using samples from patients with other laboratory-confirmed acute infections (n = 64): common cold human coronavirus (n = 20), adenovirus (n = 8), enterovirus (n = 2), influenza virus A (n = 3), and B (n = 1), parainfluenza virus 2/3 (n = 2), rhinovirus (n = 6), syncytial respiratory virus (n = 1), dengue virus (n = 20), and varicella-zoster virus (n = 1). In addition, patient samples with autoimmune antibodies (n = 51) and those collected from the general population prior to January 2020 (n = 38) were tested. Samples collected in 2020 from suspected COVID-19 patients with a negative RT-PCR result (n = 13: April, n = 11, and May, n = 2) were also included for specificity assessment. The first confirmed locally acquired COVID-19 cases in Quebec were in March 2020.

A panel to assess test sensitivity was prepared by collecting serum samples from confirmed COVID-19 patients prospectively in sufficient amounts to accommodate all serology platforms at each site. Because of the small volumes of several patient samples dedicated to the specificity panel, we opted to pool two to four samples from patients sharing the same laboratory findings. With the exception of sera obtained from patients with autoimmune diseases, all other samples were prepared by a pooling procedure.

Evaluation of serological assays.Testing of the commercial immunoassays using various reagent chemistries (enzyme immunosorbent assay [ELISA], chemiluminescence immunoassay [CLIA], electrochemiluminescence immunoassay [ECLIA], and chemiluminescent microparticles immunoassay [CMIA]) and different SARS-CoV-2 antigens (nucleocapsid, spike, and spike domains/epitope) was performed according to the manufacturer’s instructions (Table 2). Qualitative results were derived based upon the clinically based cutoff values determined by the manufacturers and described in the package inserts. All platforms used their built-in associated programmed protocols, calibrators, and controls for quality control assessment for each assay. All assays were validated to utilize both serum and plasma samples. Five assays were evaluated for the detection of IgG against SARS-CoV-2, while four assays were examined for both IgM and IgG. Finally, one assay was evaluated for the detection of IgA. All sites received the same panels of samples for performance assessment. All assays were performed in two separate sites in order to measure the interlaboratory reproducibility. Qualitative results (positive, negative, and equivocal) were inferred using the manufacturer’s specified cutoff values. Equivocal results were considered negative in the calculation of sensitivity and specificity.

View this table:
  • View inline
  • View popup
  • Download powerpoint
TABLE 2

Selected serological assays and platformsa

Three lateral-flow rapid antibody detection devices were tested per the manufacturer’s indications at only one site (site 6). All tests were designed to detect both IgM and IgG antibodies in serum or plasma samples. All results were recorded by visual inspection of reactive bands by two technologists. Quality control was assessed by the observation of a colored band on the migration matrices at the control position. A positive result was scored by one technologist when a strong colored band for either IgM and/or IgG appeared at the expected positions, while weak signals needed a consensus between two technologists for a final positive result. Invalid assays with a negative-control band were repeated until a valid result was obtained. We obtained 13 and six invalid tests for BTNX and Biocan, respectively, and none for NADAL.

Virus neutralization activity assessment of SARS-CoV-2 patient sera.Sixty-seven COVID-19 patients’ samples in our sensitivity panel were assessed for virus neutralization activity using a pseudovirus neutralization protocol described previously (11–14). Target cells were infected with single-round luciferase-expressing lentiviral particles. Briefly, 293T cells were transfected by the calcium phosphate method with the lentiviral vector pNL4.3.Luc.R-E-(NIH AIDS Reagent Program) and a plasmid encoding SARS-CoV-2 spike at a ratio of 5:4. Two days posttransfection, cell supernatants were harvested and stored at −80°C until use. 293T-ACE2 target cells were seeded at a density of 1 × 104 cells/well in 96-well luminometer-compatible tissue culture plates (Perkin Elmer) 24 h before infection. Recombinant viruses in a final volume of 100 μl were incubated with various serum dilutions (1/50, 1/250, 1/1,250, 1/6,250, and 1/31,250) for 1 h at 37°C and were then added to the target cells followed by incubation for 48 h at 37°C; cells were lysed by the addition of 30 μl of passive lysis buffer (Promega) followed by one freeze-thaw cycle. A LB942 TriStar luminometer (Berthold Technologies) was used to measure the luciferase activity of each well after the addition of 100 μl of luciferin buffer (15 mM MgSO4, 15 mM KPO4 [pH 7.8], 1 mM ATP, and 1 mM dithiothreitol) and 50 μl of 1 mM d-luciferin potassium salt (Thermo Fisher Scientific). The neutralization half-maximal inhibitory dilution (ID50) represents the serum dilution that inhibited 50% of the infection of 293T-ACE2 cells by recombinant viruses.

Calculations and statistical analyses.Contingency tables were used to compute sensitivity (Se), specificity (Sp), positive predictive values (PPV), and negative predictive values (NPV) as a function of prevalence and total agreement. All statistical analyses were calculated using GraphPad Prism version 8.4.3 (GraphPad, San Diego, CA, USA). The exact binomial proportion 95% confidence intervals (Clopper and Pearson method) were calculated for sensitivity and specificity parameters. Spearman correlation analysis was performed to assess the neutralization prediction capabilities of the commercial assays using data generated by the neutralization and enzyme immunoassays. The determination of optimized cutoff values of the serological assays was accomplished using receiver operating characteristic (ROC) curves and the Youden index. The interrater agreement (Cohen’s kappa) coefficient and total agreement were used to compare the interlaboratory reproducibility of enzyme immunoassays.

RESULTS

Clinical performances.As expected from other studies, the Se of all serological assays tested was relatively low in the first 2 weeks following the onset of symptoms and gradually increased afterwards (Table 3; also, see Fig. S1 in the supplemental material). The ELISA-based assays EuroImmun IgA and Bio-Rad total antibody were the most sensitive tests in the acute phase of COVID-19 (0 to 7 days after symptom onset) with sensitivities of 37% to 39% and 24% to 26%, respectively, while all the chemiluminescence-based assays were less sensitive in the same time frame (9% to 20%). In the 8- to 14-day period, all the assays showed increased sensitivities varying from 44% to 83%, with the ELISA-based assays still being the most sensitive (81% to 83%) compared to chemiluminescent assays. In the 15- to 34-day period, the sensitivity continued to increase for all assays (72% to 92%) with the Bio-Rad total antibody (87%), Ortho-Clinical total antibody (85% to 88%), and Euroimmun IgA (85%) being the most sensitive enzyme immunoassays.

View this table:
  • View inline
  • View popup
  • Download powerpoint
TABLE 3

Analytical performance for enzyme immunoassays and lateral-flow chromatographic assays

The maximum sensitivities observed following the 2-week acute window (≥15 days) were achieved by the Bio-Rad total-antibody assay, with an Se of 90% (confidence interval [CI], 83 to 96), and the Ortho-Clinical Diagnostics total-antibody assay, with an Se of 93% (CI, 85 to 97). The NADAL rapid antibody detection device was even more sensitive in the same time period, with an Se of 95% (CI, 88 to 98), while another rapid test, the BTNX assay, also yielded good Se (91%; CI, 84 to 96). Detection of antibodies from all COVID-19 patients was seemingly achieved 5 weeks after the onset of symptoms by only two assays, namely, the Ortho-Clinical total-antibody and NADAL IgM/IgG assays (100%; CI, 90 to 100). In fact, the NADAL rapid device was the only assay which met the minimal acceptable target Se values of ≥95% 2 weeks after the onset of symptoms set by Health Canada (https://www.canada.ca/en/health-canada/services/drugs-health-products/covid19-industry/medical-devices/testing/serological/notice-sensitivity-specificity-values.html). This level of Se was achieved only much later (≥5 weeks after symptom onset) by the other enzyme immunoassays, such as the Roche total-antibody, Bio-Rad total-antibody, Siemens total-antibody, Ortho-Clinical IgG, EuroImmun IgA, and BTNX IgM/IgG rapid tests. Antibody type-specific Se variations were also noted; for example, the EuroImmun IgA assay, although not the most sensitive overall, was the most sensitive (37 to 39%; CI, 23 to 55) of all enzyme immunoassays tested (9 to 26%) in the first week. Furthermore, the Ortho-Clinical assays showed different Se values based upon the antibody kit version, with the total-antibody assay being more sensitive at all time periods than its IgG counterpart. Interestingly, the NADAL and BTNX assays showed equivalent sensitivities to most enzyme immunoassays in all time periods. However, the NADAL assay seems to be the most sensitive of all rapid lateral-flow devices.

The Sp of all assays was assessed by testing patients diagnosed with an disease unrelated to COVID-19 (infectious or autoimmune) and patients from the pre-COVID-19 general population. The chemiluminescence-based assays (Abbott, DiaSorin, Beckman, Ortho-Clinical, Siemens, and Roche) were the most specific platforms, reaching 98% to 100% specificity. According to our results, these assays met the specificity threshold fixed at ≥98% by Health Canada. Conversely, the enzyme-linked immunosorbent assays (Bio-Rad and Euroimmun) were less successful than the former at excluding non-COVID-19 immune profiles, with Sp values ranging from 77% to 97%. Despite the fact that the Euroimmun IgA assay showed good sensitivity, it lacked Sp compared to all other tests. The rapid lateral-flow devices NADAL (98%) and Biocan (99%) had excellent specificities and met the Health Canada threshold specificity criteria.

Positive reactions were noted with samples from the pre-COVID-19 general population (n = 38) for the Bio-Rad assay (5/38) and the Euroimmun IgA (6/38) and IgG (2/38) assays (Table 4). The BTNX assay also yield a positive reaction (1/38) within this population. Of interest, three assays obtained a reactive result with one sample from the COVID-19 RT-PCR negative population (n = 13), namely, the Bio-Rad (1/13), EuroImmun IgA (3/13), and IgG (1/13) assays. No positive reactions with control populations were noted for the chemiluminescence-based platforms.

View this table:
  • View inline
  • View popup
  • Download powerpoint
TABLE 4

Cross-reactivity of enzyme immunoassays and lateral flow devices with sera from pre-COVID general population and patients diagnosed with unrelated disease

Cross-reactions with sera from patients of the other disease panel were mostly observed for the Euroimmun IgA and the BTNX assays (Table 4). The Euroimmun IgA assay was reactive against 4 of 20 seasonal coronavirus samples and 5 of 23 samples from patients diagnosed with other respiratory virus infections. The BTNX assay exhibited a lack of Sp when challenged with autoimmune sera, with 35% (18/51) of these samples yielding a positive reaction. The interference of autoantibodies was also observed for the EuroImmun IgA assay but to a lesser extent (23%; 9/39). Interestingly, nonspecific reactions were noted for six serological assays with samples containing antibodies against the dengue virus.

The PPV and NPV as a function of COVID-19 prevalence are presented in Table 5. These values were calculated using the data from ≥15 days after symptom onset. The four assays which are 100% specific (at two sites) are expected to accurately identify all COVID-19 patients at every specified prevalence level. However, the relatively small population that provided samples for our study precludes such interpretation. In fact, the 95% confidence interval calculated for 100% Sp indicates that the true value is probably between 98 and 100%. (Table 3). A decrease of only 1% in Sp has a profound effect on the PPV in a low-prevalence setting. As expected, the PPVs of all assays which are not 100% specific are quite low at 1% prevalence and increase as the prevalence rises. At 5% prevalence, the best PPV values were those of the Beckman (100%), Abbott (100%), DiaSorin (100%), Roche (100%) and Ortho-Clinical total-antibody (100%) assays, followed by Siemens (82%), Ortho-Clinical IgG (82%), and Biocan (82%).

View this table:
  • View inline
  • View popup
  • Download powerpoint
TABLE 5

Positive and negative predictive values of serological assays at ≥15 days after symptom onset as a function of COVID-19 seroprevalence

The interlaboratory reproducibility of enzyme immunoassays was assessed by the determination of the total agreement percentages and kappa values using the sample panel for sensitivity evaluation. The total agreement was ≥94% for all assays but Euroimmun IgA (90%). The kappa coefficient, which is a better statistic to compare the performance of different laboratories, yielded values between 0.763 and 1.00, which indicates very good to perfect agreement between two observers (15). The Abbott assay was the only one to reach 100% reproducibility between distant sites.

Virus-neutralizing-activity predictive ability of enzyme immunoassays.In order to determine whether the selected serological assays could detect antibodies with virus neutralization properties, we performed nonparametric correlation studies with 67 COVID-19 patient samples with titrated virus neutralization activities (ID50). Pearson’s correlation analysis was accomplished using the signal-to-cutoff (S/CO) values of the EIAs and the ID50 values obtained with the virus neutralization assay for each sample (Fig. 1). Only six samples with binding antibodies but no neutralization antibodies were found for all assays (Fig. 1, upper left quadrants of the graphs). In addition, 14 samples with no binding or neutralization antibodies were identified in all assays (Fig. 1, lower left quadrants (and 25 samples were positive for both binding and neutralization antibodies for all assays (Fig. 1, upper right quadrants). The remaining 22 samples showed different distribution patterns between assays. Spearman r correlation statistics between the serological assays and the neutralization assay are presented in Fig. 2 for each assay at both sites. No perfect correlation was found among all enzyme immunoassays and the neutralization assay (assessed by neutralization titer, i.e., the highest dilution at which the input virus signal was reduced by 50% [NT50]). However, we noted that assays designed with spike protein or domains derived from spike tend to be better correlates of neutralizing activities (0.67 to 0.76) than nucleocapsid-based assays (0.53 to 0.64). A Mann-Whitney test between paired data revealed a statistically significant difference (P = 0.0095) between spike protein and nucleocapsid-based assays (Fig. 3). The sole exception to this finding was the DiaSorin assay, which utilizes spike domains 1 and 2 (S1 and S2 domains) and was not found to be significantly different from the nucleocapsid-based assays. Correlation analyses between all immunoassays’ S/CO values were also performed. As expected, higher correlations were observed between assays using similar antigens.

FIG 1
  • Open in new tab
  • Download powerpoint
FIG 1

Evaluation of the virus neutralization predictive ability of 10 enzyme immunoassays. Spearman correlation analysis of virus neutralization ID50 and immunoassay S/CO values for 67 COVID-19 samples collected between 1 and 43 days after symptom onset. Dotted lines represent the cutoff values for each assay. Data points are presented for two testing sites.

FIG 2
  • Open in new tab
  • Download powerpoint
FIG 2

Impact of antigen type on the correlation between antibody binding activity and virus neutralization antibody titers. Spearman r values derived from correlation analysis of ID50 values versus S/CO values for enzyme immunoassays (67 paired data sets from the Se panel). Ag, antigen; N, nucleocapsid; S1, spike domain 1; S1-S2, spike domains 1 and 2; RBD, spike receptor binding domain; S, spike.

FIG 3
  • Open in new tab
  • Download powerpoint
FIG 3

Spike-based immunoassays are better predictor of virus neutralization activity. Plotting of Spearman r statistics versus antigen type used by enzyme immunoassays. **, significantly different at a P value of 0.0095; ns, not significant.

Receiver operating characteristic curve analysis.We used ROC curve analyses and the Youden index method at two different time points to optimize either the Se or the Sp. Keeping in mind that specificity and PPV are critical parameters in a low-prevalence setting, we tried to set the cutoff levels according to the minimal limit values recommended by Health Canada (Se ≥ 95%; Sp ≥ 98%; ≥15 days). In some specific clinical cases, such as multisystem inflammatory syndrome in children (16), antibody detection assays are used as adjunct tests to molecular testing for patients with a negative SARS-CoV-2 RT-PCR result (17). However, most of the serological tests are poorly sensitive in the acute phase of COVID-19. Therefore, the optimization of Se at the expense of a slight decrease in Sp could yield enhanced assays to support the diagnosis of special clinical cases.

We performed the ROC curve analyses on assays exhibiting excellent Sp. Assays with lower Sp did not have enough flexibility to increase the Se significantly, since the optimization process is a trade-off mechanism. Also, only the Ortho-Clinical total-antibody assay was analyzed, since its IgG counterpart was less sensitive. The optimized cutoff values and the calculated areas under the curves (AUC) are presented in Table 6. All ROC curves yielded excellent AUC (≥0.95) for all time points.

View this table:
  • View inline
  • View popup
  • Download powerpoint
TABLE 6

Optimization of enzyme immunoassays cutoff values using ROC curve analyses

Most of the assays are very specific (≥97%) using the company’s cutoff values, and the optimization is therefore mostly oriented toward adjusting Se. Reducing the cutoff to >0.13 for the Roche assay significantly increased the Se to 96% and, as a trade-off, resulted in a 2% loss in Sp in the ≥15-day period. All the other assays had a remarkable increase in Se (89% to 94%) without reaching the minimal value of 95%. The Bio-Rad assay exhibited a slight increase in Sp to 99% at the optimized cutoff value set at >1.2, while the Se remained at 90%.

At the ≥35-day time point, the assays from Roche, Bio-Rad, and Siemens showed good performance at the company’s cutoff values, while the Ortho-Clinical total-antibody assay was 100% sensitive and specific without any modification of the cutoff. Six of seven assays attained a Se of 100% following a modification of the cutoff values, with a slight decrease in Sp (1% to 2%) for five assays. The biggest gain in Se (+21%) was seen with the Beckman assay, which achieved 100% Se with an adjusted threshold of >0.28. Again, the Bio-Rad assay showed a 2% increase in Sp when its threshold value was slightly raised to >1.2 without any substantial loss of Se.

DISCUSSION

SARS-CoV-2 serological assays should exhibit excellent sensitivities and specificities to be useful as clinical tools for public health investigations or as a diagnostic aid in particular clinical cases. Health Canada recommends that the Se and Sp of serological assays be higher than or equal to 95% and 98%, respectively, in samples collected 2 weeks or more after the onset of symptoms. Unfortunately, none of the evaluated commercial enzyme immunoassays achieved this level of accuracy using the cutoff values suggested by the companies. Within the rapid-device category, only the NADAL assay was able to comply with the minimal requirements in this time window.

SARS-CoV-2 seroprevalence studies in low-prevalence settings such as those seen in the general population need highly specific assays to rule out potential false-positive reactions resulting from past or present unrelated infections or health conditions. Public health studies of the COVID-19 prevalence in various cohorts from distinct regions in the province of Quebec are ongoing, but preliminary data indicate a seroprevalence of 3% in blood donors 18 to 69 years old in the greater Montreal area (www.hema-quebec.qc.ca). Seven immunoassays examined in this study are predicted to have excellent specificity (98% to 100%) and good to excellent PPV (82% to 100%) when used in a low (5%)-seroprevalence setting (Table 5). Accordingly, these assays could potentially be helpful to get a more precise epidemiologic assessment of SARS-CoV-2 exposition following outbreaks in different population settings, such as nursing homes and schools, to name a few.

Similarly to other studies (18–21), we observe a direct relationship between the sensitivity rates of serological assays and time of sample collection following the onset of symptoms. Long et al. reported a median time for IgM and IgG seroconversion of 13 days after symptom onset (22). This finding is consistent with the lack of Se observed for most assays in the acute phase. Overall, we observed Se values as low as 74% and as high as 95% at ≥15 days, while this range increases to 79% to 100% at ≥5 weeks after symptom onset. According to a meta-analysis of the accuracy of serological tests, the Se of different types of serological assays was highest when they were used with samples collected at least 3 weeks after symptom onset (23). The authors reported Se values ranging from 79.7 to 98.9%, 15 days or more following the onset of symptoms.

We also noted that most of our time-stratified Se values were lower than those reported in the company package inserts. For instance, we obtained 85% and 83% sensitivities for the Roche and Abbott assays in the ≥15-day period, while the companies claimed 100% sensitivity for the same time frame. Similar to our findings, other studies reported lower Se values, varying from 73% to 89% and 82% to 92% for the Roche and Abbott assays, respectively (19–21). Clinical performance characteristic variations between serological studies are not unusual and are highly dependent on the patient cohort used for evaluation. Our patient cohort consisted essentially of symptomatic patients with limited available clinical information. Among the 176 patients, only 48 had complete clinical information, including 32 hospitalized patients and 8 patients at the intensive care unit (ICU). Therefore, it was not possible to stratify our data according to the patient setting (outpatient, ward, ICU). Furthermore, it was demonstrated in several studies that the levels of antibodies in COVID-19 patients are significantly higher in severely ill patients (24, 25). It is possible that our cohort of patients presented a moderate level of disease, which could translate to a lower level of antibody detection.

Regarding the rapid antibody detection devices, we obtained relatively good Se and Sp values which were lower than those claimed by the manufacturers but similar to those in published studies (19, 26).

Considering the low Se values reported to date, it is quite clear that serology cannot be used to diagnose acute infections and should rather be used to assess SARS-CoV-2 exposure, as recommended by public health authorities. The lack of Se documented in the acute phase of COVID-19 disease represents a yet-unresolved problem that could be bypassed by several proposed strategies. For example, an orthogonal algorithm using two consecutive enzyme immunoassays with improved global analytical accuracy (27) was proposed by the Centers for Disease Control and Prevention (28). Another path toward better clinical performance could be achieved by the development of second-generation immunoassays with recalibrated cutoff values by manufacturers, as proposed in numerous validation studies (18, 20). Collectively, the determination of optimized cutoff values in our study through ROC curve analyses had a significant impact on the diagnostic resolution of several enzyme immunoassays by increasing the sensitivity significantly without a large trade-off in specificity.

Another medical application for serology is the assessment of the immune status of convalescent-phase plasma collected from recovered SARS-CoV-2 patients. Immune-plasma therapy is considered a therapeutic option to treat very ill patients for whom there is no curative treatment (29). However, assays designed to measure the virus-neutralizing activities of antibodies are tedious and necessitate biosafety level 3 (BSL3) conditions. Alternatively, pseudovirus neutralization assays are performed in BSL2 research laboratories but cannot be implemented easily in clinical laboratories. The virus neutralization assays typically used the spike protein to assess the presence of neutralizing antibodies (nAb). The spike protein was shown to be crucial for SARS-CoV and SARS-CoV-2 to bind to angiotensin-converting enzyme 2 (ACE2) as the entry receptor on mammalian cells via the receptor binding domain (RBD) epitope of their S1 domain (30). Intuitively, serological assays which utilize spike protein should be able to detect antibodies having virus neutralization characteristics more efficiently than nucleocapsid-based assays. We titrated 67 samples from COVID-19 patients for the presence of nAb by pseudovirus neutralization assays and performed correlation statistics in order to determine whether the enzyme immunoassays routinely used in clinical settings could predict the presence of nAb in patient samples. Correlation analysis using a paired data set for each sample using the ID50 values of the neutralization assay and the signal-to-cutoff values of enzyme immunoassays indicated a moderate (nucleocapsid-based assays) to high (spike-based assays) strength of association between nAb and binding antibodies. In fact, the spike-based assays showed the highest coefficient of association (r; 0.67 to 0.77). Therefore, these results indicate that spike-based immunoassays could be useful as correlates for the detection of nAb in patient samples or alternatively to evaluate the immune status of vaccinated populations. In accordance with our data, the study by Muecksch et al. (31) showed that commercial spike-based serological assay measurements were better predictors of virus neutralization potency than results of nucleocapsid-based assays. Another study reported that commercially available serological tests have various degrees of accuracy in predicting nAb activity in convalescent-phase plasma (32). However, the authors of that study did not observe a correlation between antigen type used in serological assays and the predictability of nAb activity levels. Clearly, additional investigations, including functional and structural studies about the native conformation of the whole antigen or subdomains or the glycosylation status of the recombinant spike protein could potentially help to develop enhanced assays.

Furthermore, it will be important to develop assays sensitive enough to detect antibodies in patients enrolled in SARS-CoV-2 vaccination trials. Vaccine development research groups are mostly focusing one spike protein or the RBD, since these proteins/peptides were shown to elicit the production of potent nAb and to confer protection in nonhuman primates challenged with SARS-CoV-2 (33–35).

An important limitation of this study is related to the possible dilution effect of using a pooling procedure for the preparation of the samples for the specificity panel. Due to potential variation of antibody levels among patients, this could potentially reduce the final concentration of cross-reacting antibodies below the detection threshold for any particular kit and so increase the apparent specificity of that assay. Conversely, cross-reacting antibodies were detected in pooled or nonpooled samples (autoimmune panel) with the Euroimmun assays (IgA and IgG). In this case, other factors, such as antigen preparation (peptide conformation and glycosylation status), may contribute to the high false-positive rates observed with these assays.

In this study, we evaluated 10 enzyme immunoassays and three lateral-flow chromatography assays. The specificity was excellent for the majority of the assays, as they were probably designed to be very specific by the manufacturers at the expense of sensitivity. However, several scenarios exist to increase their clinical performance. Orthogonal algorithm testing and optimized enzyme immunoassays should be validated promptly and implemented in clinical settings. As more data accumulate, spike-based immunoassays seem to be better correlates of neutralization activity and could potentially be used to characterize plasma samples from recovered patients in routine clinical laboratories. In the face of the putative second wave of COVID, it will be important to investigate the immunity of different populations. With this knowledge at hand, it will be possible in the long term to examine the protection levels conferred by nAb in immunized individuals from natural infection or vaccination as the pandemic continues to spread.

ACKNOWLEDGMENTS

We thank all members of the SARS-CoV-2 serology provincial committee for helpful discussion about the overall logistic organization of this study. We also thank all the medical laboratory technologists for their excellent work.

We declare no conflicts of interest.

V.M.-L. is funded by the FRQS Clinical Research Scholar-Junior 1. R. Gasser is supported by a MITACS Acceleration postdoctoral fellowship.

FOOTNOTES

    • Received 28 September 2020.
    • Returned for modification 27 October 2020.
    • Accepted 8 December 2020.
    • Accepted manuscript posted online 10 December 2020.
  • Supplemental material is available online only.

  • Copyright © 2021 American Society for Microbiology.

All Rights Reserved.

This article is made available via the PMC Open Access Subset for unrestricted noncommercial re-use and secondary analysis in any form or by any means with acknowledgement of the original source. These permissions are granted for the duration of the World Health Organization (WHO) declaration of COVID-19 as a global pandemic.

REFERENCES

  1. 1.↵
    1. Wu F,
    2. Zhao S,
    3. Yu B,
    4. Chen Y-M,
    5. Wang W,
    6. Song Z-G,
    7. Hu Y,
    8. Tao Z-W,
    9. Tian J-H,
    10. Pei Y-Y,
    11. Yuan M-L,
    12. Zhang Y-L,
    13. Dai F-H,
    14. Liu Y,
    15. Wang Q-M,
    16. Zheng J-J,
    17. Xu L,
    18. Holmes EC,
    19. Zhang Y-Z
    . 2020. A new coronavirus associated with human respiratory disease in China. Nature 579:265–269. doi:10.1038/s41586-020-2008-3.
    OpenUrlCrossRefPubMed
  2. 2.↵
    1. Sironi M,
    2. Hasnain SE,
    3. Rosenthal B,
    4. Phan T,
    5. Luciani F,
    6. Shaw M-A,
    7. Sallum MA,
    8. Mirhashemi ME,
    9. Morand S,
    10. González-Candelas F
    , Editors of Infection, Genetics and Evolution. 2020. SARS-CoV-2 and COVID-19: a genetic, epidemiological, and evolutionary perspective. Infect Genet Evol 84:104384. doi:10.1016/j.meegid.2020.104384.
    OpenUrlCrossRef
  3. 3.↵
    1. Zhou P,
    2. Yang X-L,
    3. Wang X-G,
    4. Hu B,
    5. Zhang L,
    6. Zhang W,
    7. Si H-R,
    8. Zhu Y,
    9. Li B,
    10. Huang C-L,
    11. Chen H-D,
    12. Chen J,
    13. Luo Y,
    14. Guo H,
    15. Jiang R-D,
    16. Liu M-Q,
    17. Chen Y,
    18. Shen X-R,
    19. Wang X,
    20. Zheng X-S,
    21. Zhao K,
    22. Chen Q-J,
    23. Deng F,
    24. Liu L-L,
    25. Yan B,
    26. Zhan F-X,
    27. Wang Y-Y,
    28. Xiao G-F,
    29. Shi Z-L
    . 2020. A pneumonia outbreak associated with a new coronavirus of probable bat origin. Nature 579:270–273. doi:10.1038/s41586-020-2012-7.
    OpenUrlCrossRefPubMed
  4. 4.↵
    1. Zhang T,
    2. Wu Q,
    3. Zhang Z
    . 2020. Probable pangolin origin of SARS-CoV-2 associated with the COVID-19 outbreak. Curr Biol 30:1346–1351.E2. doi:10.1016/j.cub.2020.03.022.
    OpenUrlCrossRefPubMed
  5. 5.↵
    1. Corman VM,
    2. Landt O,
    3. Kaiser M,
    4. Molenkamp R,
    5. Meijer A,
    6. Chu DK,
    7. Bleicker T,
    8. Brünink S,
    9. Schneider J,
    10. Schmidt ML,
    11. Mulders DG,
    12. Haagmans BL,
    13. van der Veer B,
    14. van den Brink S,
    15. Wijsman L,
    16. Goderski G,
    17. Romette J-L,
    18. Ellis J,
    19. Zambon M,
    20. Peiris M,
    21. Goossens H,
    22. Reusken C,
    23. Koopmans MP,
    24. Drosten C
    . 2020. Detection of 2019 novel coronavirus (2019-nCoV) by real-time RT-PCR. Euro Surveill 25:2000045. doi:10.2807/1560-7917.ES.2020.25.3.2000045.
    OpenUrlCrossRefPubMed
  6. 6.↵
    1. Qian G-Q,
    2. Chen X-Q,
    3. Lv D-F,
    4. Ma AHY,
    5. Wang L-P,
    6. Yang N-B,
    7. Chen X-M
    . 2020. Duration of SARS-CoV-2 viral shedding during COVID-19 infection. Infect Dis (Lond) 52:511–512. doi:10.1080/23744235.2020.1748705.
    OpenUrlCrossRefPubMed
  7. 7.↵
    1. Zou L,
    2. Ruan F,
    3. Huang M,
    4. Liang L,
    5. Huang H,
    6. Hong Z,
    7. Yu J,
    8. Kang M,
    9. Song Y,
    10. Xia J,
    11. Guo Q,
    12. Song T,
    13. He J,
    14. Yen H-L,
    15. Peiris M,
    16. Wu J
    . 2020. SARS-CoV-2 viral load in upper respiratory specimens of infected patients. N Engl J Med 382:1177–1179. doi:10.1056/NEJMc2001737.
    OpenUrlCrossRefPubMed
  8. 8.↵
    1. To KK-W,
    2. Tsang OT-Y,
    3. Leung W-S,
    4. Tam AR,
    5. Wu T-C,
    6. Lung DC,
    7. Yip CC-Y,
    8. Cai J-P,
    9. Chan JM-C,
    10. Chik TS-H,
    11. Lau DP-L,
    12. Choi CY-C,
    13. Chen L-L,
    14. Chan W-M,
    15. Chan K-H,
    16. Ip JD,
    17. Ng AC-K,
    18. Poon RW-S,
    19. Luo C-T,
    20. Cheng VC-C,
    21. Chan JF-W,
    22. Hung IF-N,
    23. Chen Z,
    24. Chen H,
    25. Yuen K-Y
    . 2020. Temporal profiles of viral load in posterior oropharyngeal saliva samples and serum antibody responses during infection by SARS-CoV-2: an observational cohort study. Lancet Infect Dis 20:565–574. doi:10.1016/S1473-3099(20)30196-1.
    OpenUrlCrossRefPubMed
  9. 9.↵
    1. Lee S,
    2. Meyler P,
    3. Mozel M,
    4. Tauh T,
    5. Merchant R
    . 2020. Asymptomatic carriage and transmission of SARS-CoV-2: what do we know? Can J Anaesth 20:1424–1430. doi:10.1007/s12630-020-01729-x.
    OpenUrlCrossRef
  10. 10.↵
    1. Clapham H,
    2. Hay J,
    3. Routledge I,
    4. Takahashi S,
    5. Choisy M,
    6. Cummings D,
    7. Grenfell B,
    8. Metcalf CJE,
    9. Mina M,
    10. Barraquer IR,
    11. Salje H,
    12. Tam CC
    . 2020. Seroepidemiologic study designs for determining SARS-COV-2 transmission and immunity. Emerg Infect Dis 26:1978–1986. doi:10.3201/eid2609.201840.
    OpenUrlCrossRef
  11. 11.↵
    1. Beaudoin-Bussières G,
    2. Laumaea A,
    3. Anand SP,
    4. Prévost J,
    5. Gasser R,
    6. Goyette G,
    7. Medjahed H,
    8. Perreault J,
    9. Tremblay T,
    10. Lewin A,
    11. Gokool L,
    12. Morrisseau C,
    13. Bégin P,
    14. Tremblay C,
    15. Martel-Laferrière V,
    16. Kaufmann DE,
    17. Richard J,
    18. Bazin R,
    19. Finzi A
    . 2020. Decline of humoral responses against SARS-CoV-2 spike in convalescent individuals. mBio 11:e02590-20. doi:10.1128/mBio.02590-20.
    OpenUrlAbstract/FREE Full Text
  12. 12.↵
    1. Ding S,
    2. Laumaea A,
    3. Benlarbi M,
    4. Beaudoin-Bussières G,
    5. Gasser R,
    6. Medjahed H,
    7. Pancera M,
    8. Stamatatos L,
    9. McGuire AT,
    10. Bazin R,
    11. Finzi A
    . 2020. Antibody binding to SARS-CoV-2 S glycoprotein correlates with but does not predict neutralization. Viruses 12:1214. doi:10.3390/v12111214.
    OpenUrlCrossRef
  13. 13.↵
    1. Anand SP,
    2. Chen Y,
    3. Prévost J,
    4. Gasser R,
    5. Beaudoin-Bussières G,
    6. Abrams CF,
    7. Pazgier M,
    8. Finzi A
    . 2020. Interaction of human ACE2 to membrane-bound SARS-CoV-1 and SARS-CoV-2 S glycoproteins. Viruses 12:1104. doi:10.3390/v12101104.
    OpenUrlCrossRef
  14. 14.↵
    1. Prévost J,
    2. Gasser R,
    3. Beaudoin-Bussières G,
    4. Richard J,
    5. Duerr R,
    6. Laumaea A,
    7. Anand SP,
    8. Goyette G,
    9. Benlarbi M,
    10. Ding S,
    11. Medjahed H,
    12. Lewin A,
    13. Perreault J,
    14. Tremblay T,
    15. Gendron-Lepage G,
    16. Gauthier N,
    17. Carrier M,
    18. Marcoux D,
    19. Piché A,
    20. Lavoie M,
    21. Benoit A,
    22. Loungnarath V,
    23. Brochu G,
    24. Haddad E,
    25. Stacey HD,
    26. Miller MS,
    27. Desforges M,
    28. Talbot PJ,
    29. Maule GTG,
    30. Côté M,
    31. Therrien C,
    32. Serhir B,
    33. Bazin R,
    34. Roger M,
    35. Finzi A
    . 2020. Cross-sectional evaluation of humoral responses against SARS-CoV-2 spike. Cell Rep Med 1:100126. doi:10.1016/j.xcrm.2020.100126.
    OpenUrlCrossRef
  15. 15.↵
    1. Landis JR,
    2. Koch GG
    . 1977. The measurement of observer agreement for categorical data. Biometrics 33:159–174. doi:10.2307/2529310.
    OpenUrlCrossRefPubMedWeb of Science
  16. 16.↵
    1. Ahmed M,
    2. Advani S,
    3. Moreira A,
    4. Zoretic S,
    5. Martinez J,
    6. Chorath K,
    7. Acosta S,
    8. Naqvi R,
    9. Burmeister-Morton F,
    10. Burmeister F,
    11. Tarriela A,
    12. Petershack M,
    13. Evans M,
    14. Hoang A,
    15. Rajasekaran K,
    16. Ahuja S,
    17. Moreira A
    . 2020. Multisystem inflammatory syndrome in children: a systematic review. EClinicalMedicine 26:100527. doi:10.1016/j.eclinm.2020.100527.
    OpenUrlCrossRef
  17. 17.↵
    1. Tam H,
    2. Tal TE,
    3. Go E,
    4. Yeung RSM
    . 2020. Pediatric inflammatory multisystem syndrome temporally associated with COVID-19: a spectrum of diseases with many names. CMAJ 192:E1093–E1096. doi:10.1503/cmaj.201600.
    OpenUrlFREE Full Text
  18. 18.↵
    1. Bryan A,
    2. Pepper G,
    3. Wener MH,
    4. Fink SL,
    5. Morishima C,
    6. Chaudhary A,
    7. Jerome KR,
    8. Mathias PC,
    9. Greninger AL
    . 2020. Performance characteristics of the Abbott Architect SARS-CoV-2 IgG assay and seroprevalence in Boise, Idaho. J Clin Microbiol 58:e00941-20. doi:10.1128/JCM.00941-20.
    OpenUrlAbstract/FREE Full Text
  19. 19.↵
    1. Charlton CL,
    2. Kanji JN,
    3. Johal K,
    4. Bailey A,
    5. Plitt SS,
    6. MacDonald C,
    7. Kunst A,
    8. Buss E,
    9. Burnes LE,
    10. Fonseca K,
    11. Berenger BM,
    12. Schnabl K,
    13. Hu J,
    14. Stokes W,
    15. Zelyas N,
    16. Tipples G
    . 2020. Evaluation of six commercial mid to high volume antibody and six point of care lateral flow assays for detection of SARS-CoV-2 antibodies. J Clin Microbiol 58:e01361-20. doi:10.1128/JCM.01361-20.
    OpenUrlAbstract/FREE Full Text
  20. 20.↵
    1. Hörber S,
    2. Soldo J,
    3. Relker L,
    4. Jürgens S,
    5. Guther J,
    6. Peter S,
    7. Lehmann R,
    8. Peter A
    . 2020. Evaluation of three fully-automated SARS-CoV-2 antibody assays. Clin Chem Lab Med 58:2113–2120. doi:10.1515/cclm-2020-0975.
    OpenUrlCrossRef
  21. 21.↵
    1. Theel ES,
    2. Harring J,
    3. Hilgart H,
    4. Granger D
    . 2020. Performance characteristics of four high-throughput immunoassays for detection of IgG antibodies against SARS-CoV-2. J Clin Microbiol 58:e01243-20. doi:10.1128/JCM.01243-20.
    OpenUrlAbstract/FREE Full Text
  22. 22.↵
    1. Long Q-X,
    2. Liu B-Z,
    3. Deng H-J,
    4. Wu G-C,
    5. Deng K,
    6. Chen Y-K,
    7. Liao P,
    8. Qiu J-F,
    9. Lin Y,
    10. Cai X-F,
    11. Wang D-Q,
    12. Hu Y,
    13. Ren J-H,
    14. Tang N,
    15. Xu Y-Y,
    16. Yu L-H,
    17. Mo Z,
    18. Gong F,
    19. Zhang X-L,
    20. Tian W-G,
    21. Hu L,
    22. Zhang X-X,
    23. Xiang J-L,
    24. Du H-X,
    25. Liu H-W,
    26. Lang C-H,
    27. Luo X-H,
    28. Wu S-B,
    29. Cui X-P,
    30. Zhou Z,
    31. Zhu M-M,
    32. Wang J,
    33. Xue C-J,
    34. Li X-F,
    35. Wang L,
    36. Li Z-J,
    37. Wang K,
    38. Niu C-C,
    39. Yang Q-J,
    40. Tang X-J,
    41. Zhang Y,
    42. Liu X-M,
    43. Li J-J,
    44. Zhang D-C,
    45. Zhang F,
    46. Liu P,
    47. Yuan J,
    48. Li Q,
    49. Hu J-L,
    50. Chen J
    , et al. 2020. Antibody responses to SARS-CoV-2 in patients with COVID-19. Nat Med 26:845–848. doi:10.1038/s41591-020-0897-1.
    OpenUrlCrossRefPubMed
  23. 23.↵
    1. Bastos ML,
    2. Tavaziva G,
    3. Abidi SK,
    4. Campbell JR,
    5. Haraoui L-P,
    6. Johnston JC,
    7. Lan Z,
    8. Law S,
    9. MacLean E,
    10. Trajman A,
    11. Menzies D,
    12. Benedetti A,
    13. Khan FA
    . 2020. Diagnostic accuracy of serological tests for COVID-19: systematic review and meta-analysis. BMJ 370:m2516. doi:10.1136/bmj.m2516.
    OpenUrlAbstract/FREE Full Text
  24. 24.↵
    1. Young BE,
    2. Ong SWX,
    3. Ng LFP,
    4. Anderson DE,
    5. Chia WN,
    6. Chia PY,
    7. Ang LW,
    8. Mak T-M,
    9. Kalimuddin S,
    10. Chai LYA,
    11. Pada S,
    12. Tan SY,
    13. Sun L,
    14. Parthasarathy P,
    15. Fong S-W,
    16. Chan Y-H,
    17. Tan CW,
    18. Lee B,
    19. Rötzschke O,
    20. Ding Y,
    21. Tambyah P,
    22. Low JGH,
    23. Cui L,
    24. Barkham T,
    25. Lin RTP,
    26. Leo Y-S,
    27. Renia L,
    28. Wang L-F,
    29. Lye DC
    , Singapore 2019 Novel Coronavirus Outbreak Research Team. 2020. Viral dynamics and immune correlates of COVID-19 disease severity. Clin Infect Dis Epub ahead of print. doi:10.1093/cid/ciaa1280.
    OpenUrlCrossRef
  25. 25.↵
    1. Naaber P,
    2. Hunt K,
    3. Pesukova J,
    4. Haljasmägi L,
    5. Rumm P,
    6. Peterson P,
    7. Hololejenko J,
    8. Eero I,
    9. Jõgi P,
    10. Toompere K,
    11. Sepp E
    . 2020. Evaluation of SARS-CoV-2 IgG antibody response in PCR positive patients: comparison of nine tests in relation to clinical data. PLoS One 15:e0237548. doi:10.1371/journal.pone.0237548.
    OpenUrlCrossRef
  26. 26.↵
    1. Dortet L,
    2. Ronat J-B,
    3. Vauloup-Fellous C,
    4. Langendorf C,
    5. Mendels D-A,
    6. Emeraud C,
    7. Oueslati S,
    8. Girlich D,
    9. Chauvin A,
    10. Afdjei A,
    11. Bernabeu S,
    12. Pape SL,
    13. Kallala R,
    14. Rochard A,
    15. Verstuyft C,
    16. Fortineau N,
    17. Roque-Afonso A-M,
    18. Naas T
    . 2020. Evaluating ten commercially-available SARS-CoV-2 rapid serological tests using the STARD (Standards for Reporting of Diagnostic Accuracy Studies) method. medRxiv 2020.09.10.20192260.
  27. 27.↵
    1. Xu G,
    2. Emanuel AJ,
    3. Nadig S,
    4. Mehrotra S,
    5. Caddell BA,
    6. Curry SR,
    7. Nolte FS,
    8. Babic N
    . 2020. Evaluation of orthogonal testing algorithm for detection of SARS-CoV-2 IgG antibodies. Clin Chem Epub ahead of print. doi:10.1093/clinchem/hvaa210.
    OpenUrlCrossRef
  28. 28.↵
    CDC. 2020. Information for laboratories about coronavirus (COVID-19). Centers for Disease Control and Prevention, Atlanta, GA.
  29. 29.↵
    1. Farhat RM,
    2. Mousa MA,
    3. Daas EJ,
    4. Glassberg MK
    . 2020. Treatment of COVID-19: perspective on convalescent plasma transfusion. Front Med (Lausanne) 7:435. doi:10.3389/fmed.2020.00435.
    OpenUrlCrossRef
  30. 30.↵
    1. Hoffmann M,
    2. Kleine-Weber H,
    3. Schroeder S,
    4. Krüger N,
    5. Herrler T,
    6. Erichsen S,
    7. Schiergens TS,
    8. Herrler G,
    9. Wu N-H,
    10. Nitsche A,
    11. Müller MA,
    12. Drosten C,
    13. Pöhlmann S
    . 2020. SARS-CoV-2 cell entry depends on ACE2 and TMPRSS2 and is blocked by a clinically proven protease inhibitor. Cell 181:271–280.E8. doi:10.1016/j.cell.2020.02.052.
    OpenUrlCrossRefPubMed
  31. 31.↵
    1. Muecksch F,
    2. Wise H,
    3. Batchelor B,
    4. Squires M,
    5. Semple E,
    6. Richardson C,
    7. McGuire J,
    8. Cleary S,
    9. Furrie E,
    10. Greig N,
    11. Hay G,
    12. Templeton K,
    13. Lorenzi JCC,
    14. Hatziioannou T,
    15. Jenks SJ,
    16. Bieniasz P
    . 2020. Longitudinal analysis of clinical serology assay performance and neutralising antibody levels in COVID19 convalescents. medRxiv https://doi.org/10.1101/2020.08.05.20169128.
  32. 32.↵
    1. Luchsinger LL,
    2. Ransegnola B,
    3. Jin D,
    4. Muecksch F,
    5. Weisblum Y,
    6. Bao W,
    7. George PJ,
    8. Rodriguez M,
    9. Tricoche N,
    10. Schmidt F,
    11. Gao C,
    12. Jawahar S,
    13. Pal M,
    14. Schnall E,
    15. Zhang H,
    16. Strauss D,
    17. Yazdanbakhsh K,
    18. Hillyer CD,
    19. Bieniasz PD,
    20. Hatziioannou T
    . 2020. Serological assays estimate highly variable SARS-CoV-2 neutralizing antibody activity in recovered COVID19 patients. J Clin Microbiol 58:e02005-20. doi:10.1128/JCM.02005-20.
    OpenUrlAbstract/FREE Full Text
  33. 33.↵
    1. Burton DR,
    2. Walker LM
    . 2020. Rational vaccine design in the time of COVID-19. Cell Host Microbe 27:695–698. doi:10.1016/j.chom.2020.04.022.
    OpenUrlCrossRefPubMed
  34. 34.↵
    1. Yu J,
    2. Tostanoski LH,
    3. Peter L,
    4. Mercado NB,
    5. McMahan K,
    6. Mahrokhian SH,
    7. Nkolola JP,
    8. Liu J,
    9. Li Z,
    10. Chandrashekar A,
    11. Martinez DR,
    12. Loos C,
    13. Atyeo C,
    14. Fischinger S,
    15. Burke JS,
    16. Slein MD,
    17. Chen Y,
    18. Zuiani A,
    19. N Lelis FJ,
    20. Travers M,
    21. Habibi S,
    22. Pessaint L,
    23. Van Ry A,
    24. Blade K,
    25. Brown R,
    26. Cook A,
    27. Finneyfrock B,
    28. Dodson A,
    29. Teow E,
    30. Velasco J,
    31. Zahn R,
    32. Wegmann F,
    33. Bondzie EA,
    34. Dagotto G,
    35. Gebre MS,
    36. He X,
    37. Jacob-Dolan C,
    38. Kirilova M,
    39. Kordana N,
    40. Lin Z,
    41. Maxfield LF,
    42. Nampanya F,
    43. Nityanandam R,
    44. Ventura JD,
    45. Wan H,
    46. Cai Y,
    47. Chen B,
    48. Schmidt AG,
    49. Wesemann DR,
    50. Baric RS,
    51. Alter G
    , et al. 2020. DNA vaccine protection against SARS-CoV-2 in rhesus macaques. Science 369:806–811. doi:10.1126/science.abc6284.
    OpenUrlAbstract/FREE Full Text
  35. 35.↵
    1. Premkumar L,
    2. Segovia-Chumbez B,
    3. Jadi R,
    4. Martinez DR,
    5. Raut R,
    6. Markmann A,
    7. Cornaby C,
    8. Bartelt L,
    9. Weiss S,
    10. Park Y,
    11. Edwards CE,
    12. Weimer E,
    13. Scherer EM,
    14. Rouphael N,
    15. Edupuganti S,
    16. Weiskopf D,
    17. Tse LV,
    18. Hou YJ,
    19. Margolis D,
    20. Sette A,
    21. Collins MH,
    22. Schmitz J,
    23. Baric RS,
    24. de Silva AM
    . 2020. The receptor binding domain of the viral spike protein is an immunodominant and highly specific target of antibodies in SARS-CoV-2 patients. Sci Immunol 5:e02005-20. doi:10.1126/sciimmunol.abc8413.
    OpenUrlCrossRef
PreviousNext
Back to top
Download PDF
Citation Tools
Multicenter Evaluation of the Clinical Performance and the Neutralizing Antibody Activity Prediction Properties of 10 High-Throughput Serological Assays Used in Clinical Laboratories
C. Therrien, B. Serhir, M. Bélanger-Collard, J. Skrzypczak, D. K. Shank, C. Renaud, J. Girouard, V. Loungnarath, M. Carrier, G. Brochu, F. Tourangeau, B. Gilfix, A. Piche, R. Bazin, R. Guérin, M. Lavoie, V. Martel-Laferrière, C. Fortin, A. Benoit, D. Marcoux, N. Gauthier, A. M. Laumaea, R. Gasser, A. Finzi, M. Roger
Journal of Clinical Microbiology Feb 2021, 59 (3) e02511-20; DOI: 10.1128/JCM.02511-20

Citation Manager Formats

  • BibTeX
  • Bookends
  • EasyBib
  • EndNote (tagged)
  • EndNote 8 (xml)
  • Medlars
  • Mendeley
  • Papers
  • RefWorks Tagged
  • Ref Manager
  • RIS
  • Zotero
Print

Alerts
Sign In to Email Alerts with your Email Address
Email

Thank you for sharing this Journal of Clinical Microbiology article.

NOTE: We request your email address only to inform the recipient that it was you who recommended this article, and that it is not junk mail. We do not retain these email addresses.

Enter multiple addresses on separate lines or separate them with commas.
Multicenter Evaluation of the Clinical Performance and the Neutralizing Antibody Activity Prediction Properties of 10 High-Throughput Serological Assays Used in Clinical Laboratories
(Your Name) has forwarded a page to you from Journal of Clinical Microbiology
(Your Name) thought you would be interested in this article in Journal of Clinical Microbiology.
CAPTCHA
This question is for testing whether or not you are a human visitor and to prevent automated spam submissions.
Share
Multicenter Evaluation of the Clinical Performance and the Neutralizing Antibody Activity Prediction Properties of 10 High-Throughput Serological Assays Used in Clinical Laboratories
C. Therrien, B. Serhir, M. Bélanger-Collard, J. Skrzypczak, D. K. Shank, C. Renaud, J. Girouard, V. Loungnarath, M. Carrier, G. Brochu, F. Tourangeau, B. Gilfix, A. Piche, R. Bazin, R. Guérin, M. Lavoie, V. Martel-Laferrière, C. Fortin, A. Benoit, D. Marcoux, N. Gauthier, A. M. Laumaea, R. Gasser, A. Finzi, M. Roger
Journal of Clinical Microbiology Feb 2021, 59 (3) e02511-20; DOI: 10.1128/JCM.02511-20
del.icio.us logo Digg logo Reddit logo Twitter logo CiteULike logo Facebook logo Google logo Mendeley logo
  • Top
  • Article
    • ABSTRACT
    • INTRODUCTION
    • MATERIALS AND METHODS
    • RESULTS
    • DISCUSSION
    • ACKNOWLEDGMENTS
    • FOOTNOTES
    • REFERENCES
  • Figures & Data
  • Info & Metrics
  • PDF

KEYWORDS

COVID-19
SARS-CoV-2
antigen
immunoassays
neutralizing antibodies
serology

Related Articles

Cited By...

About

  • About JCM
  • Editor in Chief
  • Board of Editors
  • Editor Conflicts of Interest
  • For Reviewers
  • For the Media
  • For Librarians
  • For Advertisers
  • Alerts
  • RSS
  • FAQ
  • Permissions
  • Journal Announcements

Authors

  • ASM Author Center
  • Submit a Manuscript
  • Article Types
  • Resources for Clinical Microbiologists
  • Ethics
  • Contact Us

Follow #JClinMicro

@ASMicrobiology

       

ASM Journals

ASM journals are the most prominent publications in the field, delivering up-to-date and authoritative coverage of both basic and clinical microbiology.

About ASM | Contact Us | Press Room

 

ASM is a member of

Scientific Society Publisher Alliance

 

American Society for Microbiology
1752 N St. NW
Washington, DC 20036
Phone: (202) 737-3600

 

Copyright © 2021 American Society for Microbiology | Privacy Policy | Website feedback

Print ISSN: 0095-1137; Online ISSN: 1098-660X