ABSTRACT
Quantitative measurement of anti-HBs is used to evaluate the response to hepatitis B vaccination in health care workers and to optimize postexposure management. The different guidelines for hepatitis B vaccination and booster policy imply that the measurement of anti-HBs levels by different assays is accurate and consistent, yielding comparable quantitative results. We measured anti-HBs levels in 200 serum samples from patients and health care professionals by nine different anti-HBs assays and compared the quantitative results and the performance characteristics of the different test systems. The assay specificity ranged between 96.8 and 100% when sera from individuals without a vaccination history and with negative anti-HBc status were defined as true negatives. Sensitivity ranged between 93.5 and 100%. A high number of sera showed discrepancies between measurements by the different systems. The mean coefficient of variation between the different measurements was 47.1% (range, 15.0 to 201.0%), and the factors of multiplication ranged from 2.8 to 105. Hemolysis or lipemia did not seem to influence the measurement, and there was no difference between anti-HBc-positive and -negative individuals. The classical enzyme immunoassays tend to find lower anti-HBs levels than the automated systems, with higher values by the Abbott AXSYM assay. The serial dilution of the international standard preparation was measured accurately by most of the assays. In conclusion, the quantitative measurement of anti-HBs levels is not reliable, even though an international standard is used for the calibration of the systems. Some systems showed specific problems that should be addressed by the manufacturers.
The antibody response to hepatitis B virus surface antigen (anti-HBs) is an important serological marker for vaccine-induced immunity to hepatitis B virus (HBV). An adequate vaccine response is defined as an anti-HBs level of ≥100 IU/liter 4 to 8 weeks after the last of three or four vaccine injections. It is widely accepted that a sustained level of at least 10 IU/liter is protective against HBV infection. Vaccinees without sufficient anti-HBs responses, so-called nonresponders or low responders, undergo a special regimen of additional vaccine doses. For liver transplant recipients, quantitative measurement of anti-HBs levels is used in the management of hepatitis B immune globulin prophylaxis, which is initiated to maintain anti-HBs levels of at least 100 or 200 IU/liter, according to different guidelines. All these recommendations imply that the measurement of anti-HBs levels by different assays is accurate and consistent, yielding comparable quantitative results in various laboratories and countries. However, analysis of routine clinical samples by different systems revealed significant discrepancies for a number of sera. This observation is in accordance with older reports comparing outdated methods such as the radioimmunoassay and latex agglutination (9, 12, 17, 19). The aims of this study were to determine whether the quantification of anti-HBs levels in routine clinical samples by different test systems is comparable and accurate within acceptable limits and to identify factors contributing to the variability of test results. In addition, we evaluated the ease of use, time to results, and general practicability of the systems as requirements for integration into the workflow of modern high-throughput medical laboratories. To our knowledge, this report provides the first systematic and comprehensive comparison of currently marketed anti-HBs assays.
MATERIALS AND METHODS
Test samples.Two hundred serum samples from patients and health care workers were taken from daily submissions for routine anti-HBs testing without any preselection. A sample was included in the study if at least 2 ml of serum was available. Retrospectively, 145 serum samples came from individuals with histories of vaccination, and 24 other samples were anti-HBc positive; 122 were from health care professionals, and 78 were from patients, 20 of whom were immunosuppressed. A pool comprising 20 sera with anti-HBs levels measuring around 100 IU/liter by our routine anti-HBs assay was used for the evaluation of intra- and interassay variability. The first reference preparation of hepatitis B immune globulin, distributed by CLB (Amsterdam, The Netherlands), was measured in different dilutions to check the calibration of the assays. The standard preparation was diluted in an anti-HBs negative-control serum (Dade Behring, Marburg, Germany) to concentrations between 500 and 7.5 IU/liter. A pool of anti-HBs-negative lipemic sera and a pool of hemolytic sera were spiked with 100 IU/liter of the standard preparation and tested by each system. Five selected serum samples were serially diluted in negative-control serum for measurement of the linearity of dilution.
Assay systems.Six different automated immunoassays (the Abbott Axsym AUSAB and five chemiluminescence assays) and three enzyme immunoassays (EIAs) were performed.
(i) Abbott Axsym AUSAB assay.The Abbott Axsym assay is a microparticle EIA using recombinant HBsAg (ad/ay) on microparticles as the solid phase and biotin coupled to recombinant HBsAg as the conjugate. In the next step, alkaline phosphatase-conjugated anti-biotin is bound to the antigen sandwich. The reaction mixture is transferred to an inert glass fiber matrix to which the microparticles bind irreversibly. Methylumbelliferyl phosphate is used as a substrate, and the fluorescence of the final product, methylumbelliferone, is read by the instrument.
(ii) Chemiluminescence assays.The following five chemiluminescence assays were used: the Advia Centaur anti-HBs assay on the Advia Centaur system from Bayer Diagnostics (now part of Siemens Medical Solutions, Fernwald, Germany), the Vitros anti-HBs assay on the Vitros ECI Immunodiagnostic system (Ortho Clinical Diagnostics, Raritan, NJ), the Roche Elecsys anti-HBs assay on the Modular System (Roche Diagnostics, Basel, Switzerland), the Liaison anti-HBs assay on the Liaison system (DiaSorin, Turin, Italy), and the Abbott Architect anti-HBs assay on the Architect i2000 system (Abbott, Chicago, IL). The technical specifications of the assays are summarized in Table 1. Serum samples with values beyond the linearity of the test system were diluted as recommended by the manufacturers. Most of the automats have an autodilute function; in the others, a special dilution buffer for these purposes is provided, and dilution protocols are included in the software.
Characteristics of chemiluminescence assays
(iii) EIAs.The following three EIAs were used as recommended by the manufacturers: the ETI-AB-AUK-3 assay (DiaSorin), the Enzygnost anti-HBs assay (Dade Behring), and the Monolisa anti-HBs assay (Bio-Rad Laboratories, Redmond, WA). The EIAs were performed on the Behring ELISA Processor III using the protocol recommended by the manufacturer. Samples with results beyond the linearity of the standard curve were diluted 1:20 and/or 1:100 in negative-control serum. All the EIAs use human plasma-derived HBsAg (ad/ay) and are sandwich immunoassays using a tetramethylbenzidine substrate. The Behring assay uses a one-point calibration method (alpha method); the highest values measurable without dilution ranged from 170 to 240 IU/liter. The Bio-Rad and DiaSorin assays use four calibrators, from 10 to 150 IU/liter and from 10 to 1,000 IU/liter, respectively.
Statistical analysis.Statistical analysis was performed with SPSS for Windows (version 13.0) and Medcalc. Because there is no “gold standard” for the measurement of anti-HBs levels, we defined sera as true positive or true negative if they were identified as positive (≥10 IU/liter) or negative (<10 IU/liter) by ≥6 of 9 assays. We also calculated specificity by using sera from individuals without vaccination histories that were negative for anti-HBc. Bland-Altman analysis was used to compare the assays with each other and against the geometric mean value for all assays. Inter-rater agreement was used to compare the rating of the assays. Receiver operating characteristic (ROC) analysis was performed by postulating that sera with anti-HBs levels of ≥10 IU/liter by ≥6 assays are true positives.
RESULTS
Performance of assays.The performance characteristics of the six automated test systems are summarized in Table 2. The hands-on-time for manual order entry is relatively high in most of the systems, especially if the samples are not bar coded. The system with the easiest and quickest sample order entry is the DiaSorin Liaison; the most time-consuming systems are the Advia Centaur and the Abbott Axsym. The Roche Elecsys, Abbott Architect, and Ortho ECI systems are capable of diluting samples with >1,000 IU/liter automatically, while the DiaSorin Liaison system needs a special kit for automatic dilution (Anti-HBs Plus). For the Abbott Axsym and Advia Centaur assays, sera must be diluted manually in dilution buffer (not included in the kit), but the results are calculated automatically. The systems require a minimum of 150 to 200 μl of serum for test performance. Low sample volumes create annoying problems in some systems. The Advia Centaur and Abbott Architect systems discard the serum when they aspirate air with the fluid. Because the systems retry the aspiration, there is often no sample left. The Liaison system performs the assay even if a low volume is detected. The result will get an error flag, or in some cases, there will be no result at the end of the run. The Ortho ECI and Abbott Axsym systems measure the level in the sample cup and will not perform the assay if the level detected is too low.
Performance characteristics of automated test systems
The performance characteristics of the three EIAs are shown in Table 3.
Performance characteristics of EIAs
The DiaSorin EIA kit comes with a negative control and four calibrators between 10 and 1,000 IU/liter. A 100-μl volume of incubation buffer is pipetted into all wells before addition of samples and controls/calibrators. The manufacturer recommends remeasuring sera with optical densities of >3.000 at 405 nm and calculating corrected values if exact quantification is desired.
The Behring EIA kit comes with a negative control and one standard using the alpha method for quantification. The highest anti-HBs levels measurable by this method are relatively low (170 to 240 IU/liter in our runs), so many sera have to be diluted further for exact quantification (70 of 200 sera of our series had to be diluted). This results in higher test costs and a longer time to results.
The Bio-Rad EIA kit comes with a negative control and one standard for qualitative measurement. For quantitative measurement, a standard kit has to be ordered separately (at extra cost). The highest standard is 150 IU/liter, so Bio-Rad recommends measuring all sera both undiluted and at a dilution of 1:10. When the Bio-Rad EIA was used as recommended by the manufacturer and the anti-HBs standard kit was used for quantification, the values obtained for all sera and for the dilutions of the international standard were much lower than those obtained by the other assays (Table 4). After excluding handling errors, we measured the standards of the standard kit by three different systems and found out that the concentrations of the standards were much higher than specified (Table 5). We therefore calibrated the assay with the international standard in dilutions from 7.5 to 500 IU/liter. The results obtained by this method seemed plausible and were therefore used for further analysis.
Measurement of the international standard using calibration with Bio-Rad standards
Measurement of Bio-Rad standards with other test systems
Levels of anti-HBs in samples.For 36 serum samples, the level of anti-HBs was determined to be <10 IU/liter by all nine assays. At least six of the nine assays determined anti-HBs levels to be <10 IU/liter in a total of 47 serum samples. The distribution of positive values is given in Table 6.
Distribution of positive values
Specificity and sensitivity.Thirty-one serum samples were from individuals who had never received hepatitis B vaccine and who tested negative for anti-HBc. By taking these sera as true negatives for the calculation of specificity, five assays showed a specificity of 100% and four assays yielded a low-positive value for one serum sample each (specificity, 96.8%; range of false-positive values, 10.3 to 14.7 IU/liter). At least six of the nine assays found anti-HBs levels below 10 IU/liter in a total of 47 sera. The specificities calculated with these sera and the range of anti-HBs levels determined for each false-positive serum sample are shown in Tables 7 and 8, respectively. Three of the serum samples with positive values by single systems were anti-HBc positive, and eight were from individuals with histories of vaccination.
Specificities of test systems
Properties of false-positive samples
For 139 sera, anti-HBs levels of ≥10 IU/liter were obtained by the majority of test systems. When sensitivity was calculated with these sera, five assays determined anti-HBs levels to be ≥10 IU/liter in 100% of the sera, and the other four assays found levels below 10 IU/liter in two to nine sera, as shown in Tables 9 and 10.
Sensitivities of test systems
Properties of false-negative samples
Quantification.A high number of serum samples showed large discrepancies between results from different systems. The mean coefficient of variation (CV) between the different measurements was 47.1%. For 57.6% of the sera with positive values in the majority of assays, the CV between the nine measurements was greater than 47%; the factor of multiplication ranged from 2.8 to 105 for these sera. The range of discrepant values for some of the sera with the highest CVs is shown in Table 11. When only one or two assays showed discrepant values, the measurement was repeated by these assays, but all values were confirmed. When serum samples were classified according to the anti-HBs levels determined by the majority of assays (10 to 100, 101 to 1,000, and >1,000 IU/liter), the mean CV was significantly lower for the group of sera with anti-HBs levels between 101 and 1,000 IU/liter than for sera with levels between 10 and 100 or above 1,000 IU/liter. (36.2% versus 57.4% and 60.3%, respectively; P = 0.003). However, there were serum samples with extreme differences in all three groups.
Examples of sera with very high discrepancies between measurements
The mean CV for anti-HBc-positive sera was not significantly different from that for anti-HBc-negative sera, and the mean CV for the sera of patients did not differ significantly from that for sera from health care professionals.
By assuming that all sera were taken 4 to 8 weeks after vaccination and by using the actual definitions of low antibody response as levels between 10 and 100 IU/liter and of good response as levels above 100 IU/liter, 27.5% of the vaccinees would have been sorted into different groups by different assays. Twenty-nine sera showed anti-HBs levels below 10 IU/liter by some assays and above 10 IU/liter by others.
When the anti-HBs levels obtained by the assays were compared with each other and with the geometric mean titers for all assays by using Bland-Altman analysis (2), the Abbott Axsym assay in most cases showed higher levels and the EIAs showed lower levels than the automated chemiluminescence assays. The mean bias for the Axsym assay and the DiaSorin EIA, for example, was 56.8% (Fig. 1). It can be seen from Fig. 1 that results for many serum samples differed by more than 100%. The bias was also high when systems with recombinant antigen but different test platforms were compared with each other. A lower bias was seen when systems using similar platforms (e.g., the Bayer Centaur and Abbott Architect systems; the Behring, DiaSorin, and Bio-Rad EIAs) were compared, but results for many serum samples showed very high discrepancies among these test systems as well. The calculated biases are shown in Table 12.
Bland-Altman (difference [expressed as a percentage] versus average [expressed in IU per liter]) plot of Abbott Axsym and DiaSorin EIA results (values of <1,000 IU/liter).
Bland-Altman analysis of assay systems compared with each other and with geometric mean titers
When the assay systems were compared not by analyzing the absolute values but by inter-rater agreement classifying the sera as either negative, ≥10 IU/liter, >100 IU/liter, or >1,000 IU/liter, the strength of agreement (kappa) ranged from 0.650 to 0.879 (lower 95% confidence interval, <0.610) (Table 13). Kappa values of 0.610 and lower are interpreted as moderate agreement, which means that the reliability of rating is relatively low.
Inter-rater agreement between the assay systemsa
The serial dilution of the international standard preparation was measured accurately by most of the systems (except for the Bio-Rad EIA, as mentioned above), but some of the assays tended to measure the 500-IU/liter standard too high (Fig. 2).
Measurement of the international standard preparation.
Interassay variability was within an acceptable range for all of the assay systems, with values between 0.7% (Ortho ECI) and 13.5% (DiaSorin EIA). The different systems found different anti-HBs levels for the serum pool (Fig. 3).
Interassay variability, determined with pooled sera. A serum pool was analyzed five times in repeat tests on different days.
Anti-HBs levels obtained for the lipemic and hemolytic serum pools spiked with 100 IU/liter of the international standard were 80 to 120 IU/liter each; thus, no influence of lipemia or hemolysis on the quantification could be shown in this analysis.
To find out whether interferences such as matrix effects (21) could be the cause of discrepant results, we measured anti-HBs levels in serial dilutions of three sera with highly discrepant results and, for comparison, in two sera with low CVs. The results of these measurements were highly confusing, since there were three different effects of dilution: for one serum sample, the variance decreased with higher dilutions; for the second, the difference remained stable; and for the third, the difference actually increased with higher dilutions. The dilution of one of two sera with low CVs also showed higher differences with higher dilutions.
ROC analysis.To find out whether the cutoff of 10 IU/liter is valid for all assay systems, we performed a ROC analysis, presuming that all sera determined to be positive by at least six systems were true positives and all sera determined to be negative by at least six systems were true negatives. The resulting cutoff levels for the different systems are given in Table 14.
ROC analysis
DISCUSSION
Current recommendations for hepatitis B immunization of health care professionals and other groups at risk of HBV exposure include anti-HBs testing after completion of vaccination and different management of individuals with negative or low-positive (10 to 100 IU/liter) and high-positive (≥100 IU/liter) titers. Similarly, guidelines for postexposure prophylaxis require quantitative anti-HBs testing in certain cases. The recommendations of several countries are shown in Table 15 (3-5, 7, 11, 13, 15, 18, 20). Most recommendations are based on the assumption that the measurements by different assay systems are sufficiently standardized by the use of an international standard preparation. However, as shown by our study, this is obviously not the case.
Recommendations for hepatitis B revaccination of persons at increased risk in different countries
The standard preparation was produced in 1977 for the quantification of anti-HBs in immune globulin preparations. The assays available at that time differ completely from those used nowadays. The study group involved in the production of this preparation noted high discrepancies in measurements by some test systems even in these first analyses, and they decided to exclude discrepant values from further evaluation (1).
In several studies comparing different assay systems, similar discrepancies were found. The authors postulated diversity of antigens, problems with low-avidity antibodies, and different vaccine antigens to explain the discrepancies (8, 10, 12, 17, 19, 22). Different calculation methods have also been shown to cause problems for standardized quantification (22). However, not all sera showed discrepant values, and some sera showed extremely high discrepancies. In our study, sera with extremely high discrepancies were mainly from vaccinated healthy individuals, most of whom had been vaccinated with the same antigen preparation. Thus, different vaccine antigens and different calculation formulas did not seem to be essential for these differences. The antigens used in the assay systems did not seem to be responsible, either, since systems using the same antigen source also showed large differences for these sera. However, antigens may be produced in different ways, so the possibility that antigen preparation is one of the reasons for the differences cannot be excluded. Diversity in the individual immune response (proportion of low-avidity antibodies, immunoglobulin G [IgG] subclasses, etc.) and interference by endogenous proteins or other substances in the individual samples are other possible explanations for the phenomenon (21, 23).
Twenty-nine of the 200 randomly chosen serum samples showed negative values for anti-HBs in some assays and positive values in others. The range of positive values was 10.1 to 171 IU/liter. In one case (171 IU/liter by the Advia Centaur), dilution did not result in a linear curve; the values remained high. A nonspecific result is therefore probable in this case. In a second case, there was a wide range of different values between 7.8 and 135 IU/liter; the serum sample was from a vaccinated health care professional who had acquired a needle stick injury from an unknown donor. Dilution of this serum sample resulted in values between 10 and 60 IU/liter. The values declined with dilution in systems with a higher initial measurement and increased in those with a lower or negative initial measurement. Some sort of matrix effect might be instrumental in this behavior, and it seems probable that the serum is true positive.
Four of the assay systems tested in our study showed specific problems that should be addressed by the manufacturers. The standard lot of the Bio-Rad EIA was obviously not produced with the usual quality control measurements. The concentrations of the standards were much higher than those specified on the vials. The DiaSorin EIA found anti-HBs levels below 10 IU/liter for nine sera, while most of the other systems found positive values. ROC analysis resulted in a cutoff of 5 IU/liter for this system, and Bland-Altman analysis showed that this system yielded significantly lower values than most of the other assay systems. Additionally, dilution of one of the serum samples with an initial negative value led to a positive result. The Abbott Axsym assay found anti-HBs levels above 10 IU/liter for six sera, while most of the other systems found negative values. ROC analysis resulted in a cutoff of 19.1 IU/liter for this system, and Bland-Altman analysis showed a clear tendency of this system to find higher anti-HBs levels than other assay systems. The Advia Centaur assay had a clearly nonspecific result of 171 IU/liter for one serum sample. Dilution of this sample did not result in a linear curve; instead, the values remained high. None of the other systems found anti-HBs levels above 5 IU/liter for this sample. This high nonspecific value raises concerns about the specificity of the system.
Thus, our study shows that levels of anti-HBs determined by one assay system cannot be compared with those determined by other systems, although all the assays are calibrated with the same international standard. Because there is no gold standard assay, the question which test is right and which is wrong remains open.
The reasons for this are manifold. The international standard preparations, initially, were not produced for the standardization of antibody assays but for the measurement of specific antibody content in immune globulin preparations (1, 16). Most of the actual standards are themselves immune globulin products and not sera. Since the 1980s, the manufacturers of quantitative immunoassays have used such products in establishing standard curves to achieve comparability of quantitative values in different assays. However, from the beginnings of quantitative measurements, discrepancies have been noticed with different assay systems. We have seen similar problems with rubella IgG and varicella-zoster IgG (unpublished data).
Regarding the user friendliness of the test systems, all automated assays are easy to use and fast, if samples are bar coded and the order is entered by the laboratory computer system. Manual sample identification and order entry are relatively time consuming and complicated in all systems. The manufacturers should improve the software for easier handling of such samples. Better management of low-volume samples would also be desirable.
In conclusion, the immune response to specific antigens is a complex system with high variability between individuals, since the antibodies are directed against a variety of epitopes in variable concentrations. It is probably impossible to standardize the quantification using completely different assay systems. What does this mean for measuring anti-HBs levels after immunization against hepatitis B? An anti-HBs concentration of ≥10 IU/liter is assumed to be protective against both acute and chronic disease (14). Thus, a level of at least 10 IU/liter determined 4 to 8 weeks after the last injection of the basic course of immunization (usually the third dose) is regarded as proof of response to vaccination. However, this value is generally the lower limit of assay accuracy in the different systems; in fact, according to our results, five of nine tests show a calculated cutoff value above 10 IU/liter (as high as 19.1 IU/liter in one case). Thus, the risk of false-positive as well as false-negative results is relatively high. A false-negative result is more or less harmless, because it only leads to (unnecessary) revaccination. A false-positive result, on the other hand—meaning that a nonresponder is, by mistake, considered a responder—is dangerous, since the necessary revaccination will not be performed. Therefore, some countries set the lower limit of the anti-HBs level proving a good response to vaccination at 100 IU/liter (Table 15); individuals with lower levels determined 4 to 8 weeks after the third vaccination receive an additional dose. Another possibility would be regular administration of a fourth vaccine dose at month 12, which may result in a higher proportion of vaccine responders (6). Several countries recommend a fourth vaccine dose in high-risk settings, as well as a booster dose after 5 years (4, 7, 11).
The problem of false positives still remains in countries lacking such recommendations (e.g., the United States) and in all situations where actions to be taken (e.g., giving a booster dose) depend on whether the anti-HBs level determined is below or equal to/above 10 IU/liter. This might be the case for vaccinated individuals at high risk for infection who had never been tested after vaccination, and for whom it is necessary to determine whether they are immune or not, e.g., after exposure to HBV. In such a case, an anti-HBs level of only 10 IU/liter or higher would be proof of successful vaccination and probable protection. However, in view of the fact that in our study the range of positive values for sera testing positive in <4 assays (i.e., possibly false positive) was 10.1 to 19.1 IU/liter, a result of 10 IU/liter is likely to be inaccurate with the tests presently available. The ROC analysis resulted in cutoffs between 5 and 19.1 IU/liter for the different assays. Thus, the introduction of a gray zone between 5 and 20 IU/liter should be considered. This would lead to a cutoff value of 20 IU/liter for proven seropositivity. Vaccinated individuals with anti-HBs levels below 20 IU/liter should receive a booster dose and be tested 4 weeks later in order to find out whether they are true responders. The impact of quantitative anti-HBs testing in other, different contexts (e.g., following liver transplantation for HBV-infected recipients) has to be discussed separately.
FOOTNOTES
- Received 18 December 2007.
- Returned for modification 24 January 2008.
- Accepted 29 January 2008.
- Copyright © 2008 American Society for Microbiology
