Previous Article | Next Article ![]()
Journal of Clinical Microbiology, April 2008, p. 1298-1306, Vol. 46, No. 4
0095-1137/08/$08.00+0 doi:10.1128/JCM.02430-07
Copyright © 2008, American Society for Microbiology. All Rights Reserved.

Department of Virology, Freiburg University Medical Center, Freiburg,1 Institute of Medical Microbiology and Hygiene, University Hospital Regensburg, Regensburg, Germany2
Received 18 December 2007/ Returned for modification 24 January 2008/ Accepted 29 January 2008
|
|
|---|
|
|
|---|
100 IU/liter 4 to 8 weeks after the last of three or four vaccine injections. It is widely accepted that a sustained level of at least 10 IU/liter is protective against HBV infection. Vaccinees without sufficient anti-HBs responses, so-called nonresponders or low responders, undergo a special regimen of additional vaccine doses. For liver transplant recipients, quantitative measurement of anti-HBs levels is used in the management of hepatitis B immune globulin prophylaxis, which is initiated to maintain anti-HBs levels of at least 100 or 200 IU/liter, according to different guidelines. All these recommendations imply that the measurement of anti-HBs levels by different assays is accurate and consistent, yielding comparable quantitative results in various laboratories and countries. However, analysis of routine clinical samples by different systems revealed significant discrepancies for a number of sera. This observation is in accordance with older reports comparing outdated methods such as the radioimmunoassay and latex agglutination (9, 12, 17, 19). The aims of this study were to determine whether the quantification of anti-HBs levels in routine clinical samples by different test systems is comparable and accurate within acceptable limits and to identify factors contributing to the variability of test results. In addition, we evaluated the ease of use, time to results, and general practicability of the systems as requirements for integration into the workflow of modern high-throughput medical laboratories. To our knowledge, this report provides the first systematic and comprehensive comparison of currently marketed anti-HBs assays. |
|
|---|
Assay systems. Six different automated immunoassays (the Abbott Axsym AUSAB and five chemiluminescence assays) and three enzyme immunoassays (EIAs) were performed.
(i) Abbott Axsym AUSAB assay. The Abbott Axsym assay is a microparticle EIA using recombinant HBsAg (ad/ay) on microparticles as the solid phase and biotin coupled to recombinant HBsAg as the conjugate. In the next step, alkaline phosphatase-conjugated anti-biotin is bound to the antigen sandwich. The reaction mixture is transferred to an inert glass fiber matrix to which the microparticles bind irreversibly. Methylumbelliferyl phosphate is used as a substrate, and the fluorescence of the final product, methylumbelliferone, is read by the instrument.
(ii) Chemiluminescence assays. The following five chemiluminescence assays were used: the Advia Centaur anti-HBs assay on the Advia Centaur system from Bayer Diagnostics (now part of Siemens Medical Solutions, Fernwald, Germany), the Vitros anti-HBs assay on the Vitros ECI Immunodiagnostic system (Ortho Clinical Diagnostics, Raritan, NJ), the Roche Elecsys anti-HBs assay on the Modular System (Roche Diagnostics, Basel, Switzerland), the Liaison anti-HBs assay on the Liaison system (DiaSorin, Turin, Italy), and the Abbott Architect anti-HBs assay on the Architect i2000 system (Abbott, Chicago, IL). The technical specifications of the assays are summarized in Table 1. Serum samples with values beyond the linearity of the test system were diluted as recommended by the manufacturers. Most of the automats have an autodilute function; in the others, a special dilution buffer for these purposes is provided, and dilution protocols are included in the software.
|
View this table: [in a new window] |
TABLE 1. Characteristics of chemiluminescence assays
|
Statistical analysis.
Statistical analysis was performed with SPSS for Windows (version 13.0) and Medcalc. Because there is no "gold standard" for the measurement of anti-HBs levels, we defined sera as true positive or true negative if they were identified as positive (
10 IU/liter) or negative (<10 IU/liter) by
6 of 9 assays. We also calculated specificity by using sera from individuals without vaccination histories that were negative for anti-HBc. Bland-Altman analysis was used to compare the assays with each other and against the geometric mean value for all assays. Inter-rater agreement was used to compare the rating of the assays. Receiver operating characteristic (ROC) analysis was performed by postulating that sera with anti-HBs levels of
10 IU/liter by
6 assays are true positives.
|
|
|---|
|
View this table: [in a new window] |
TABLE 2. Performance characteristics of automated test systems
|
|
View this table: [in a new window] |
TABLE 3. Performance characteristics of EIAs
|
The Behring EIA kit comes with a negative control and one standard using the alpha method for quantification. The highest anti-HBs levels measurable by this method are relatively low (170 to 240 IU/liter in our runs), so many sera have to be diluted further for exact quantification (70 of 200 sera of our series had to be diluted). This results in higher test costs and a longer time to results.
The Bio-Rad EIA kit comes with a negative control and one standard for qualitative measurement. For quantitative measurement, a standard kit has to be ordered separately (at extra cost). The highest standard is 150 IU/liter, so Bio-Rad recommends measuring all sera both undiluted and at a dilution of 1:10. When the Bio-Rad EIA was used as recommended by the manufacturer and the anti-HBs standard kit was used for quantification, the values obtained for all sera and for the dilutions of the international standard were much lower than those obtained by the other assays (Table 4). After excluding handling errors, we measured the standards of the standard kit by three different systems and found out that the concentrations of the standards were much higher than specified (Table 5). We therefore calibrated the assay with the international standard in dilutions from 7.5 to 500 IU/liter. The results obtained by this method seemed plausible and were therefore used for further analysis.
|
View this table: [in a new window] |
TABLE 4. Measurement of the international standard using calibration with Bio-Rad standards
|
|
View this table: [in a new window] |
TABLE 5. Measurement of Bio-Rad standards with other test systems
|
|
View this table: [in a new window] |
TABLE 6. Distribution of positive values
|
|
View this table: [in a new window] |
TABLE 7. Specificities of test systems
|
|
View this table: [in a new window] |
TABLE 8. Properties of false-positive samples
|
10 IU/liter were obtained by the majority of test systems. When sensitivity was calculated with these sera, five assays determined anti-HBs levels to be
10 IU/liter in 100% of the sera, and the other four assays found levels below 10 IU/liter in two to nine sera, as shown in Tables 9 and 10. |
View this table: [in a new window] |
TABLE 9. Sensitivities of test systems
|
|
View this table: [in a new window] |
TABLE 10. Properties of false-negative samples
|
|
View this table: [in a new window] |
TABLE 11. Examples of sera with very high discrepancies between measurements
|
By assuming that all sera were taken 4 to 8 weeks after vaccination and by using the actual definitions of low antibody response as levels between 10 and 100 IU/liter and of good response as levels above 100 IU/liter, 27.5% of the vaccinees would have been sorted into different groups by different assays. Twenty-nine sera showed anti-HBs levels below 10 IU/liter by some assays and above 10 IU/liter by others.
When the anti-HBs levels obtained by the assays were compared with each other and with the geometric mean titers for all assays by using Bland-Altman analysis (2), the Abbott Axsym assay in most cases showed higher levels and the EIAs showed lower levels than the automated chemiluminescence assays. The mean bias for the Axsym assay and the DiaSorin EIA, for example, was 56.8% (Fig. 1). It can be seen from Fig. 1 that results for many serum samples differed by more than 100%. The bias was also high when systems with recombinant antigen but different test platforms were compared with each other. A lower bias was seen when systems using similar platforms (e.g., the Bayer Centaur and Abbott Architect systems; the Behring, DiaSorin, and Bio-Rad EIAs) were compared, but results for many serum samples showed very high discrepancies among these test systems as well. The calculated biases are shown in Table 12.
![]() View larger version (24K): [in a new window] |
FIG. 1. Bland-Altman (difference [expressed as a percentage] versus average [expressed in IU per liter]) plot of Abbott Axsym and DiaSorin EIA results (values of <1,000 IU/liter).
|
|
View this table: [in a new window] |
TABLE 12. Bland-Altman analysis of assay systems compared with each other and with geometric mean titers
|
10 IU/liter, >100 IU/liter, or >1,000 IU/liter, the strength of agreement (kappa) ranged from 0.650 to 0.879 (lower 95% confidence interval, <0.610) (Table 13). Kappa values of 0.610 and lower are interpreted as moderate agreement, which means that the reliability of rating is relatively low. |
View this table: [in a new window] |
TABLE 13. Inter-rater agreement between the assay systemsa
|
![]() View larger version (15K): [in a new window] |
FIG. 2. Measurement of the international standard preparation.
|
![]() View larger version (32K): [in a new window] |
FIG. 3. Interassay variability, determined with pooled sera. A serum pool was analyzed five times in repeat tests on different days.
|
To find out whether interferences such as matrix effects (21) could be the cause of discrepant results, we measured anti-HBs levels in serial dilutions of three sera with highly discrepant results and, for comparison, in two sera with low CVs. The results of these measurements were highly confusing, since there were three different effects of dilution: for one serum sample, the variance decreased with higher dilutions; for the second, the difference remained stable; and for the third, the difference actually increased with higher dilutions. The dilution of one of two sera with low CVs also showed higher differences with higher dilutions.
ROC analysis. To find out whether the cutoff of 10 IU/liter is valid for all assay systems, we performed a ROC analysis, presuming that all sera determined to be positive by at least six systems were true positives and all sera determined to be negative by at least six systems were true negatives. The resulting cutoff levels for the different systems are given in Table 14.
|
View this table: [in a new window] |
TABLE 14. ROC analysis
|
|
|
|---|
100 IU/liter) titers. Similarly, guidelines for postexposure prophylaxis require quantitative anti-HBs testing in certain cases. The recommendations of several countries are shown in Table 15 (3-5, 7, 11, 13, 15, 18, 20). Most recommendations are based on the assumption that the measurements by different assay systems are sufficiently standardized by the use of an international standard preparation. However, as shown by our study, this is obviously not the case. |
View this table: [in a new window] |
TABLE 15. Recommendations for hepatitis B revaccination of persons at increased risk in different countries
|
In several studies comparing different assay systems, similar discrepancies were found. The authors postulated diversity of antigens, problems with low-avidity antibodies, and different vaccine antigens to explain the discrepancies (8, 10, 12, 17, 19, 22). Different calculation methods have also been shown to cause problems for standardized quantification (22). However, not all sera showed discrepant values, and some sera showed extremely high discrepancies. In our study, sera with extremely high discrepancies were mainly from vaccinated healthy individuals, most of whom had been vaccinated with the same antigen preparation. Thus, different vaccine antigens and different calculation formulas did not seem to be essential for these differences. The antigens used in the assay systems did not seem to be responsible, either, since systems using the same antigen source also showed large differences for these sera. However, antigens may be produced in different ways, so the possibility that antigen preparation is one of the reasons for the differences cannot be excluded. Diversity in the individual immune response (proportion of low-avidity antibodies, immunoglobulin G [IgG] subclasses, etc.) and interference by endogenous proteins or other substances in the individual samples are other possible explanations for the phenomenon (21, 23).
Twenty-nine of the 200 randomly chosen serum samples showed negative values for anti-HBs in some assays and positive values in others. The range of positive values was 10.1 to 171 IU/liter. In one case (171 IU/liter by the Advia Centaur), dilution did not result in a linear curve; the values remained high. A nonspecific result is therefore probable in this case. In a second case, there was a wide range of different values between 7.8 and 135 IU/liter; the serum sample was from a vaccinated health care professional who had acquired a needle stick injury from an unknown donor. Dilution of this serum sample resulted in values between 10 and 60 IU/liter. The values declined with dilution in systems with a higher initial measurement and increased in those with a lower or negative initial measurement. Some sort of matrix effect might be instrumental in this behavior, and it seems probable that the serum is true positive.
Four of the assay systems tested in our study showed specific problems that should be addressed by the manufacturers. The standard lot of the Bio-Rad EIA was obviously not produced with the usual quality control measurements. The concentrations of the standards were much higher than those specified on the vials. The DiaSorin EIA found anti-HBs levels below 10 IU/liter for nine sera, while most of the other systems found positive values. ROC analysis resulted in a cutoff of 5 IU/liter for this system, and Bland-Altman analysis showed that this system yielded significantly lower values than most of the other assay systems. Additionally, dilution of one of the serum samples with an initial negative value led to a positive result. The Abbott Axsym assay found anti-HBs levels above 10 IU/liter for six sera, while most of the other systems found negative values. ROC analysis resulted in a cutoff of 19.1 IU/liter for this system, and Bland-Altman analysis showed a clear tendency of this system to find higher anti-HBs levels than other assay systems. The Advia Centaur assay had a clearly nonspecific result of 171 IU/liter for one serum sample. Dilution of this sample did not result in a linear curve; instead, the values remained high. None of the other systems found anti-HBs levels above 5 IU/liter for this sample. This high nonspecific value raises concerns about the specificity of the system.
Thus, our study shows that levels of anti-HBs determined by one assay system cannot be compared with those determined by other systems, although all the assays are calibrated with the same international standard. Because there is no gold standard assay, the question which test is right and which is wrong remains open.
The reasons for this are manifold. The international standard preparations, initially, were not produced for the standardization of antibody assays but for the measurement of specific antibody content in immune globulin preparations (1, 16). Most of the actual standards are themselves immune globulin products and not sera. Since the 1980s, the manufacturers of quantitative immunoassays have used such products in establishing standard curves to achieve comparability of quantitative values in different assays. However, from the beginnings of quantitative measurements, discrepancies have been noticed with different assay systems. We have seen similar problems with rubella IgG and varicella-zoster IgG (unpublished data).
Regarding the user friendliness of the test systems, all automated assays are easy to use and fast, if samples are bar coded and the order is entered by the laboratory computer system. Manual sample identification and order entry are relatively time consuming and complicated in all systems. The manufacturers should improve the software for easier handling of such samples. Better management of low-volume samples would also be desirable.
In conclusion, the immune response to specific antigens is a complex system with high variability between individuals, since the antibodies are directed against a variety of epitopes in variable concentrations. It is probably impossible to standardize the quantification using completely different assay systems. What does this mean for measuring anti-HBs levels after immunization against hepatitis B? An anti-HBs concentration of
10 IU/liter is assumed to be protective against both acute and chronic disease (14). Thus, a level of at least 10 IU/liter determined 4 to 8 weeks after the last injection of the basic course of immunization (usually the third dose) is regarded as proof of response to vaccination. However, this value is generally the lower limit of assay accuracy in the different systems; in fact, according to our results, five of nine tests show a calculated cutoff value above 10 IU/liter (as high as 19.1 IU/liter in one case). Thus, the risk of false-positive as well as false-negative results is relatively high. A false-negative result is more or less harmless, because it only leads to (unnecessary) revaccination. A false-positive result, on the other hand—meaning that a nonresponder is, by mistake, considered a responder—is dangerous, since the necessary revaccination will not be performed. Therefore, some countries set the lower limit of the anti-HBs level proving a good response to vaccination at 100 IU/liter (Table 15); individuals with lower levels determined 4 to 8 weeks after the third vaccination receive an additional dose. Another possibility would be regular administration of a fourth vaccine dose at month 12, which may result in a higher proportion of vaccine responders (6). Several countries recommend a fourth vaccine dose in high-risk settings, as well as a booster dose after 5 years (4, 7, 11).
The problem of false positives still remains in countries lacking such recommendations (e.g., the United States) and in all situations where actions to be taken (e.g., giving a booster dose) depend on whether the anti-HBs level determined is below or equal to/above 10 IU/liter. This might be the case for vaccinated individuals at high risk for infection who had never been tested after vaccination, and for whom it is necessary to determine whether they are immune or not, e.g., after exposure to HBV. In such a case, an anti-HBs level of only 10 IU/liter or higher would be proof of successful vaccination and probable protection. However, in view of the fact that in our study the range of positive values for sera testing positive in <4 assays (i.e., possibly false positive) was 10.1 to 19.1 IU/liter, a result of 10 IU/liter is likely to be inaccurate with the tests presently available. The ROC analysis resulted in cutoffs between 5 and 19.1 IU/liter for the different assays. Thus, the introduction of a gray zone between 5 and 20 IU/liter should be considered. This would lead to a cutoff value of 20 IU/liter for proven seropositivity. Vaccinated individuals with anti-HBs levels below 20 IU/liter should receive a booster dose and be tested 4 weeks later in order to find out whether they are true responders. The impact of quantitative anti-HBs testing in other, different contexts (e.g., following liver transplantation for HBV-infected recipients) has to be discussed separately.
Published ahead of print on 6 February 2008. ![]()
|
|
|---|
| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Copyright © 2009 by the American Society for Microbiology. For an alternate route to Journals.ASM.org, visit: http://intl-journals.asm.org | More Info»