Clinical Chemistry
HOME HELP FEEDBACK SUBSCRIPTIONS ARCHIVE SEARCH TABLE OF CONTENTS
 QUICK SEARCH:   [advanced]


     


Clinical Chemistry 51: 1845-1853, 2005. First published August 11, 2005; 10.1373/clinchem.2005.052381
This Article
Right arrow Abstract Freely available
Right arrow Full Text (PDF)
Right arrow All Versions of this Article:
clinchem.2005.052381v1
51/10/1845    most recent
Right arrow Alert me when this article is cited
Right arrow Alert me if a correction is posted
Right arrow Citation Map
Services
Right arrow Email this article to a friend
Right arrow Similar articles in this journal
Right arrow Similar articles in Web of Science
Right arrow Similar articles in PubMed
Right arrow Alert me to new issues of the journal
Right arrow Download to citation manager
Right arrow reprints & permissions
Citing Articles
Right arrow Citing Articles via HighWire
Right arrow Citing Articles via Web of Science (34)
Right arrow Citing Articles via Google Scholar
Google Scholar
Right arrow Articles by Waterboer, T.
Right arrow Articles by Pawlita, M.
Right arrow Search for Related Content
PubMed
Right arrow PubMed Citation
Right arrow Articles by Waterboer, T.
Right arrow Articles by Pawlita, M.
Related Collections
Right arrow Automation and Analytical Techniques
(Clinical Chemistry. 2005;51:1845-1853.)
© 2005 American Association for Clinical Chemistry, Inc.


Automation and Analytical Techniques

Multiplex Human Papillomavirus Serology Based on In Situ–Purified Glutathione S-Transferase Fusion Proteins

Tim Waterboer1,a, Peter Sehr1,1, Kristina M. Michael1, Silvia Franceschi2, John D. Nieland3, Thomas O. Joos4, Markus F. Templin4 and Michael Pawlita1

1 Infection and Cancer Program, German Cancer Research Center (DKFZ), Heidelberg, Germany.
2 Infections and Cancer Epidemiology, International Agency for Research on Cancer (IARC), Lyon, France.
3 MediGene AG, Martinsried, Germany.
4 NMI Natural and Medical Sciences Institute at the University of Tübingen, Reutlingen, Germany.

aAddress correspondence to this author at: Infection and Cancer Program, German Cancer Research Center (DKFZ), Im Neuenheimer Feld 242, 69120 Heidelberg, Germany. Fax 49-6221-42-4932; e-mail t.waterboer{at}dkfz.de.


   Abstract
Top
Abstract
Introduction
Materials and Methods
Results
Discussion
References
 
Background: More than 100 different human papillomaviruses (HPVs) can cause proliferative diseases, many of which are malignant, such as cervical cancer. HPV serology is complex because infection and disease lead to distinct type-specific antibody responses. Using bead-based technology, we have developed an assay platform that allows the simultaneous detection of antibodies against up to 100 in situ affinity–purified recombinant HPV proteins.

Methods: Twenty-seven HPV proteins were expressed as glutathione S-transferase fusion proteins and affinity-purified in one step by incubation of glutathione-displaying beads in bacterial lysate. Spectrally distinct bead sets, each carrying one particular antigen, were mixed, incubated with serum, and differentiated in a flow cytometer-like analyzer (xMAP; Luminex Corp). Antibodies bound to the antigens were detected via fluorescent secondary reagents. We studied 756 sera from 2 case-control studies of cervical cancer.

Results: Glutathione S-transferase fusion proteins bound with high affinity to glutathione-displaying beads (Kd = 6.9 x 10–9 mol/L). The dynamic range of multiplex serology covered 1.5 orders of magnitude, and antibodies were detected at serum dilutions >1:1 000 000. Imprecision (median CV) was ≤5.4%, and assay reproducibility was high (R2 = 0.97). Results on clinical samples showed high concordance with ELISA ({kappa} = 0.846), but multiplex serology exhibited increased detection of weak antibody responses. Antibodies to the E6 oncoproteins of the rare HPV types 52 and 58 were associated with cervical cancer (P <0.001).

Conclusion: Multiplex serology enables antibody analyses of large numbers of sera against up to 100 antigens in parallel and has the potential to replace ELISA technology.


   Introduction
Top
Abstract
Introduction
Materials and Methods
Results
Discussion
References
 
A method that allows the simultaneous analysis of large numbers of serum samples (i.e., >500) for antibodies against multiple viral antigens would be useful for seroepidemiologic studies on prevalence and disease association of human papillomaviruses (HPVs). 2 HPV serology is complex for several reasons. More than 100 different types of HPV can infect the epithelia of skin or mucosa (1) and induce proliferative diseases(2). Fifteen high-risk types have been identified that can cause malignant tumors, predominantly cancer of the cervix of the uterus; HPV type 16 alone accounts for more than 50% of cervical cancer cases worldwide (3). HPV antibodies are type specific(4). Those targeting the major viral capsid protein L1 are markers of infection, and those targeting the viral oncoproteins E6 and E7 are markers for HPV-associated cancer (5)(6).

Conventional serologic methods such as ELISA allow the analysis of sera for antibodies to only 1 antigen per well. Different multiplexing technologies for the analysis of protein–protein interactions have been explored (7), all depending on prepurified proteins. High-density planar arrays allow the analysis of very large numbers of targets but are limited in the number of samples that can be analyzed in a reasonable time frame at acceptable costs. Low-density suspension arrays use distinct populations of color- or size-coded beads, which serve as solid supports for the targets and can be identified in a flow cytometer. In contrast to planar arrays, bead arrays allow the analysis of large sample numbers in high-throughput fashion with a limited number of targets (8).

One such suspension array (xMAP; Luminex Corp.) uses polystyrene beads with a diameter of 5.6 µm. The beads are filled with 2 fluorescent dyes in various ratios to produce an array of 100 different bead sets that can be distinguished by their internal "color". The Luminex analyzer resembles a flow cytometer with 2 lasers. Beads of different sets mixed in the wells of a 96-well plate are aspirated and arranged in single file before passing through a detection chamber. Excitation with the red laser allows identification of the bead set, and the green laser is used to excite the reporter fluorescent label bound to each bead.

This system is an open platform for the analysis of molecular interactions and has been used for the detection of bacterial rRNA (9), cytokines(10), and single-nucleotide polymorphisms (11). Several assays have been developed for screening of monoclonal antibodies (12) and for detection of antibodies against viral antigens (13)(14)(15)(16), bacterial toxins (17), polysaccharides(18), or autoantigens (19) in serum, cerebrospinal fluid(16), dried blood-spot specimens (15), and stool samples (20). In these reports, prepurified proteins were coupled directly to the beads, and only a few antigens were analyzed in parallel.

We have developed a method for multiplex HPV serologic analysis that combines this fluorescent bead array with a generic method allowing in situ affinity purification of any glutathione S-transferase (GST) fusion protein developed for conventional ELISA (21)(22). We investigated key performance characteristics and the feasibility of this technique for large and complex seroepidemiologic studies by parallel testing for antibodies against 27 antigens (E6, E7, and L1 proteins of 9 HPV types) in more than 800 human serum samples.


   Materials and Methods
Top
Abstract
Introduction
Materials and Methods
Results
Discussion
References
 
hardware and software
Measurements were performed on a Luminex 100 Total System comprising the Luminex 100 analyzer, Luminex XYP plate handler, Luminex SD sheath fluid delivery system, a Pentium 4 personal computer (Dell) running Windows 2000 (Microsoft Corp.), and Luminex IS 2.2 SP1 software. Data analyses and graphical representations were performed with Microsoft Excel 2002 and SigmaPlot 8.0 (SPSS Inc.), respectively.

production of glutathione–casein
Glutathione ({gamma}-L-glutamyl-L-cysteinylglycine) is a tripeptide with an irregular NH2 terminus caused by the orientation of its glutamine residue. To produce glutathione–casein (GC), we cross-linked the amines of the 11 lysine residues in casein with the thiol residues of the cysteines in glutathione. Coupling to the primary amines of the glutathione must be avoided because they are involved in the interaction with GST.

The production of GC has been described (21). Briefly, after the cysteine residue of casein (cat. no. C-5890; Sigma-Aldrich) was blocked with N-ethylmaleimide (Sigma-Aldrich), the heterobifunctional cross-linker sulfo-succinimidyl-4-(P-maleimidophenyl)-butyrate (Pierce) was added to yield thiol-reactive casein-maleimidophenyl-butyrate. The latter was reacted with glutathione to yield GC.

direct coupling of gc and streptavidin-r-phycoerythrin to luminex beads
To produce GC beads, terminal amines of the GC were coupled to the carboxyl groups of the beads (MultiAnalyte and, later, SeroMAP microspheres; Luminex). After a standard carbodiimide/succinimide-based activation procedure (23), the bead carboxyls formed an acyl amino ester that reacted with the primary amines of some of the glutathione residues in the GC, yielding a stable amide bond. After this coupling procedure, at least one of the glutathione residues cross-linked the casein and the beads. Because the chemical activation procedure took place on the beads, unreacted amines of the glutathione were unaffected and still able to interact with GST.

In detail, 1.25 x 107 beads were washed twice with and finally suspended in 400 µL of activation buffer (0.1 mol/L sodium phosphate, pH 6.2). Washing was performed as follows: centrifugation for 2 min at 15 700g, aspiration of the supernatant, addition of the activation buffer, sonification for 1 min in the dark, and resuspension of the pellet with a vortex mixer. Both N-(3-dimethylaminopropyl)-N'-ethylcarbodiimide (Pierce) and N-hydroxysuccinimide (Pierce) were dissolved at a concentration of 50 g/L in activation buffer and dry dimethyl sulfoxide (Fluka), respectively, and 50 µL of each solution was immediately added to the beads. The suspension was incubated on a shaker for 20 min at room temperature in the dark. The activated beads were washed twice with 1 mL of coupling buffer (50 mmol/L 2-morpholinoethanesulfonic acid, pH 5.0) and resuspended in 1 mL of a 250 mg/L GC solution in coupling buffer. Coupling was carried out on a shaker for 2 h at room temperature in the dark. After the coupling procedure, the beads were incubated for 15 min in 1 mL of washing buffer [phosphate-buffered saline (PBS), 0.5 mL/L Tween 20, pH 7.4] containing 50 mmol/L Tris to block unreacted carboxyl groups with primary amines. The beads were then washed twice with 1 mL of washing buffer. Beads were stored in 500 µL of storage buffer (PBS, pH 7.4, containing 1 g/L casein and 0.5 g/L sodium azide) at 4 °C in the dark. Typical recovery rates were 70% for MultiAnalyte and 60% for SeroMAP microspheres, respectively, yielding a sufficient amount of beads to test at least 2500 sera.

With the same protocol, 8 different concentrations (350 to 3 mg/L; 1:2 serial dilutions) of streptavidin-R-phycoerythrin (Molecular Probes) were coupled to yield directly fluorescence-labeled beads. Before the coupling procedure, the streptavidin-R-phycoerythrin was loaded on a PD10 desalting column (Amersham Pharmacia Biotech) and eluted with PBS to remove azide.

in situ affinity purification of antigens on gc beads
Viral antigens were expressed with pGEX vectors (Amersham) in Escherichia coli as double fusion proteins with N-terminal GST and a C-terminal peptide (tag) consisting of the 11 C-terminal amino acids from the large T antigen of simian virus 40. The expression constructs for E6, E7, and L1 of HPV types 16 and 18 as GST fusion proteins have been described (21)(22). Those for the homologous proteins of HPV types 6b, 31, 33, 35, 45, 52, and 58 were generated in the same fashion. L1 proteins were expressed in E. coli strain BL21 Rosetta (Novagen).

A fusion protein consisting of GST and tag without intervening viral antigen was used for background determinations. Overexpression of GST fusion proteins was induced by addition of 250 µmol/L isopropyl-ß-D-thiogalactoside to the bacterial culture. Bacterial cells were harvested by centrifugation, and the cells were lysed in a high-pressure homogenizer (Avestin). The lysate was cleared from insoluble components by centrifugation (30 min at 4 °C and 24 500g in a SA-600 rotor), and the supernatant was mixed 1:1 with glycerol and stored at –20 °C.

Bacterial lysate was diluted to 1 g/L in casein buffer (1 g/L casein in PBS, pH 7.4). For each antigen, 3000 GC beads per serum were loaded with GST fusion proteins directly in the lysate and incubated for 1 h at room temperature in the dark on a shaker. The beads were then washed 3 times with 1 mL of casein buffer.

human sera
We analyzed 756 sera from 2 case-control studies on cervical cancer from Chennai (India) (24) and Algiers (Algeria) (25) and 72 sera from a German HPV vaccination trial [Nieland JD, et al. Vaccination trial with HPV16 L1E7 chimeric virus-like particles (CVLP) in high grade CIN patients. Manuscript in preparation]. Ethics committee approval and informed consent of study participants for HPV serology were obtained.

Sera were preincubated at a 1:50 dilution on a shaker for 1 h at room temperature in a serum preincubation buffer based on casein buffer and additionally containing 2 g/L lysate from bacteria expressing GST alone to block antibodies directed against residual bacterial proteins and GST. To suppress nonspecific binding of antibodies to the beads themselves, serum preincubation buffer also contained 5 g/L polyvinyl alcohol (cat. no. P8136; Sigma-Aldrich), 8 g/L polyvinylpyrrolidone (cat. no. PVP-360; Sigma-Aldrich), and 25 mL/L Super ChemiBlock (Chemicon).

multiplex assay
Bead sets carrying different antigens were mixed, and 50 µL each of preincubated diluted serum and mixed beads (3000 per set) were combined in 96-well plates with filter bottoms (cat. no. MSBVN12; Millipore) and incubated on a shaker for 1 h at room temperature in the dark. The beads were washed 3 times in 100 µL of casein buffer on a vacuum manifold (Millipore). Biotinylated secondary antibody [goat anti-human IgA, IgM, IgG (H+L); Dianova] diluted 1:1000 in casein buffer was added and incubated as before. After washing, detection conjugate (streptavidin-R-phycoerythrin) diluted 1:1000 in casein buffer was incubated with the beads for 30 min. The beads were washed again, and the wells were filled with casein buffer. Reporter fluorescence of the beads was determined with the Luminex analyzer and expressed as median fluorescence intensity (MFI) of at least 100 beads per set per well. To calculate antigen-specific reactivity, the MFI of GSTtag was subtracted from the antigen MFI.

Typically, for studies with up to 1000 sera, GC beads were loaded with antigen on day 1, serum dilutions and all binding reactions and washes were performed on day 2 (assay day), and the completed assay plates were stored overnight at 4 °C and analyzed on day 3. For larger studies, the number of assay days would have to be increased.

quantification of bound antigen
pGEX-derived fusion protein binding on beads was quantified with anti-tag-PE, a monoclonal murine antibody specific for the C-terminal tag epitope (21)(26). Affinity-purified antibody was directly labeled with R-phycoerythrin (PE; Chromaprobe) (27). Briefly, a solution of PE in 60% ammonium sulfate was dialyzed extensively. The amino groups of PE were reacted with the heterobifunctional cross-linker succinimidyl 4-(N-maleimidomethyl)cyclohexane-1-carboxylate (Pierce) to yield maleimide-labeled PE. The hinge disulfide bonds of the antibody were reduced by addition of dithiothreitol to yield free sulfhydryls. Finally, the maleimide-labeled PE was reacted with the reduced antibody, and unreacted sulfhydryls were blocked with N-ethylmaleimide.

gst capture elisa
The GST capture ELISA has been described in detail (21). Briefly, 96-well plates were coated overnight with GC and incubated with antigen lysate. Preincubated sera were added to the wells with immobilized antigen, and bound antibodies were detected with a horseradish peroxidase–labeled secondary antibody, with tetramethylbenzidine and H2O2 as substrate.

assay principle
Our multiplex HPV serology method used in situ affinity–purified viral antigens developed for a conventional GST capture ELISA (21)(22). For this ELISA, antigens were expressed as GST fusion proteins in E. coli and directly purified from bacterial lysates via GC (Fig. 1 , left scheme). The capture protein adsorbed to the wells of a 96-well plate, which were then filled with bacterial lysate. The interaction of glutathione and GST immobilized the fusion proteins. After the wells were washed, sera were added to the wells. The binding of specific antibodies to the antigens was detected with a horseradish peroxidase–coupled secondary antibody.



View larger version (21K):
[in this window]
[in a new window]
 
Figure 1. Schematic comparison of GST capture ELISA (left) and bead-based multiplex serology (right).

By covalently coupling the GC to the beads, we transferred the in situ affinity purification to the multiplexing system (Fig. 1Up , right scheme). GC beads were loaded with antigen directly from bacterial lysate, and unbound protein was removed by washing. Differently colored beads carrying different viral antigens were mixed and reacted with serum. Biotinylated secondary antibody and streptavidin-R-phycoerythrin were used as reporters to detect bound serum antibodies.


   Results
Top
Abstract
Introduction
Materials and Methods
Results
Discussion
References
 
affinity of gst–glutathione interaction on beads
In multiplex serology, binding of antigens is mediated by the interaction between the GST domain of the fusion proteins and glutathione on the GC beads. One initial concern was that this interaction might not be strong enough to prevent antigen carryover after beads loaded with different antigens were mixed.

We therefore incubated antigen-loaded and unloaded GC beads of different sets together overnight before antigen detection with anti-tag-PE. Antigen-loaded GC beads showed strong signals with more than 5000 MFI, but unloaded GC beads remained unchanged and, with signals <25 MFI, showed only the autofluorescence of the beads (data not shown).

This experimental exclusion of antigen carryover among GC beads is in line with the high affinity we determined for the interaction of GST and GC beads. To determine the dissociation constant, Kd, we incubated GC beads with serial dilutions of purified GSTtag. Bound GSTtag was quantified by anti-tag-PE. The GST concentration at half-maximal binding was 6.9 x 10–9 mol/L (Fig. 2 ). This affinity is comparable to that of many antibody–antigen interactions (28).



View larger version (11K):
[in this window]
[in a new window]
 
Figure 2. Determination of the GST–glutathione dissociation constant.

GC beads were incubated with a serial dilution (1:2) of purified GSTtag, and bound GSTtag was quantified by anti-tag-PE. Data points represent mean values of 2 independent experiments; error bars represent SD. Data points were fitted by use of a 2-parameter single rectangular hyperbola model and SigmaPlot software. Dashed lines indicate the half-maximal signal and the corresponding GST concentration, respectively. Because Kd = ([GST] x [glutathione])/[GST–glutathione], it can be deduced that at the GST concentration corresponding to half maximal signal [GST]1/2, one-half of the glutathione is free and one-half has bound to GST. Thus [glutathione] = [GST–glutathione], and therefore, Kd = [GST]1/2, indicating a Kd value of 6.9 x 10–9 mol/L for the interaction of GST with GC beads.

density of different gst fusion proteins on gc beads
Frequently, expression of low–molecular-weight proteins in bacteria is much better than that of larger proteins, as shown for GST fusion proteins of HPV E6 and E7 vs L1 (21)(22). We showed that in situ affinity purification on GC beads partially compensates for differences in expression efficiency of GST fusion proteins by incubating GC beads with serial dilutions of bacterial lysates containing HPV 16 E6 (46.8 kDa), E7 (38.6 kDa), and L1 (82.7 kDa) GST fusion proteins, respectively. Bound antigen was quantified by anti-tag-PE. At a lysate concentration of 1 g/L, all proteins reached saturation on the GC beads, with plateaus for E6 and E7 approximately twice as high as that for L1 (Fig. 3 ). The ~3-fold difference in lysate concentration at half-maximal binding indicated that the L1 lysate contained up to 3-fold less fusion protein than the E6 or E7 lysates. There are two possible reasons for the ~2-fold lower plateau for the L1 protein: Expression of larger GST fusion proteins may produce more N-terminal fragments that compete for glutathione binding but lack the C-terminal tag (21)(22), and/or the higher molecular weight may produce a lower molar density of fusion proteins on the bead surface. We concluded that in situ affinity purification of different GST fusion proteins on GC beads leads to similar antigen densities.



View larger version (16K):
[in this window]
[in a new window]
 
Figure 3. Binding curve of GST fusion proteins on GC beads.

GC beads were incubated with serial dilutions (1:3) of 3 bacterial lysates, and bound antigen was quantified with anti-tag-PE. Data points represent mean values of 2 independent experiments; error bars represent SD. Data points were fitted by a 4-parameter logistic curve model and SigmaPlot software. Dashed lines indicate the lysate concentrations at half-maximal signals (4.7 mg/L for HPV 16 E6, 5.7 mg/L for E7, 14.2 mg/L for L1).

dynamic range and detection limit of multiplex serology
We incubated HPV 16 E6–loaded GC beads with serial dilutions of 1 previously characterized strongly antibody-positive and 4 antibody-negative human sera to assess the dynamic range and detection limit of multiplex serology (Fig. 4 ). The linear part of the half-logarithmic dose–response curve of the positive serum covered ~1.5 orders of magnitude. However, the range allowing differentiation of negative and positive sera was far wider. Even at a 1:3 000 000 dilution, the signal of the positive serum was still significantly higher than the mean plus 3 SD of the 4 negative sera.



View larger version (14K):
[in this window]
[in a new window]
 
Figure 4. Determination of dynamic range of multiplex serology.

An HPV 16 E6 antibody-positive cervical cancer serum (•) and 4 antibody-negative sera from healthy blood donors were serially diluted (1:2), and the HPV 16 E6 reactivities were determined. Data points were fitted by a 4-parameter logistic curve model and SigmaPlot software. {circ} represent the mean values of the 4 negative sera; bars represent 3 SD, and fitting was omitted because it overlayed with the error bars. The inset shows the data at high dilutions with MFI at enlarged scale.

reproducibility of multiplex serology
We determined the imprecision of the measurement by analyzing directly fluorescence-labeled beads according to the Clinical and Laboratory Standards Institute (formerly NCCLS) guidelines (29). Within-run CVs were 0.0%–3.1% (median, 2.2%), between-run, within-day CVs were 4.2%–7.2% (median, 4.9%), and between-day CVs were 3.5%–7.5% (median, 5.4%).

We also determined assay reproducibility by comparing 2 experiments performed as independently as possible. Sera from 38 cervical cancer patients were analyzed for HPV 16 E6 and E7 antibodies by 2 different investigators, each using the same bacterial lysates as an antigen source but different sets of GC beads from separate coupling batches. For the 2 data sets, the correlation coefficient (R2) was 0.97, indicating low individual variation (Fig. 5 ). The slope of the trend line was also 0.97, indicating low systematic variation in both experiments.



View larger version (16K):
[in this window]
[in a new window]
 
Figure 5. Reproducibility of multiplex serology.

Two different experimenters loaded different sets of GC beads from separate coupling batches with antigens from the same bacterial lysates and analyzed 38 sera from cervical cancer patients for HPV 16 E6 and E7 antibodies. The linear curve fit of the 2 data sets follows the equation: y = 0.97x + 145 [SD of the slope and intercept = 0.02 (P <0.0001) and 130 (P = 0.266), respectively]. The correlation coefficient (R2) is 0.97, the standard error of the estimate is 853, and the dotted lines represent the 95% confidence interval.

After the assays were completed, multiplex serology signals were stable for at least 24 h when plates were stored at 4 °C. Signals were also stable when antigen-loaded GC beads in casein buffer were stored for 48 h at 4 °C before the serology assay. Signal loss of 15% was observed when GC beads loaded with L1 proteins of 13 HPV types were stored for 4 weeks in casein buffer containing 0.5 g/L azide (data not shown).

agreement with gst capture elisa
We directly compared the performance of multiplex serology with GST capture ELISA. Sera from 91 cervical cancer patients were analyzed for HPV 16 E6 antibodies by both methods (Fig. 6 ). The correlation coefficient, 0.94, indicated good agreement of the 2 data sets. According to cutoff values derived from healthy controls tested with both methods, 38 sera were concordantly positive, 46 were concordantly negative, and 7 sera showed discordant results, all 7 being negative by GST capture ELISA and positive by multiplex serology, but at very low titers. The {kappa} value, 0.846, demonstrated that serum classification by both assays is highly correlated.



View larger version (18K):
[in this window]
[in a new window]
 
Figure 6. Correlation of multiplex serology and GST capture ELISA.

HPV 16 E6 antibody reactivities of 91 cervical cancer patients were determined with both methods. The linear curve fit follows the equation: y = 0.07x + 45 [SD of the slope and the intercept = 0.002 (P <0.0001) and 18 (P = 0.017), respectively]. The correlation coefficient (R2) for the linear fit is 0.94, the standard error of the estimate is 139, and the dotted lines represent the 95% confidence interval. For both assays, cutoff values [dashed lines; 400 MFI and 180 milliabsorbance units at 450 nm (mAU450), respectively] were derived from the HPV 16 E6 reactions of 21 healthy blood donors (cutoff = mean + 3 SD). Sera were classified as antibody positive or negative; discordant sera are represented by gray circles. A {kappa} value of 0.846 (95% confidence interval, 0.736–0.956) was calculated from the fourfold table (30).

sensitivity
We determined assay sensitivity by reanalyzing sera from a clinical vaccination trial. Chimeric virus-like particles consisting of HPV 16 L1/E7 fusion proteins had been administered in 2 different doses to women with HPV 16 DNA-positive cervical intraepithelial neoplasia, a cervical cancer precursor lesion. The vaccination had induced strong L1-specific and mostly weak E7-specific antibody responses. The availability of pre- and postvaccination sera of women who received the vaccination and women in the placebo group allowed comparison of the sensitivity of multiplex serology to GST capture ELISA.

Increased postvaccination E7 antibodies in the high-dose group, and less often also in the low-dose group, were found with both methods, but the increase seen with multiplex serology, quantified by the ratio of postvaccination to prevaccination antibody reactivity in serum, was much stronger (Fig. 7 ). For individual vaccine recipients, the ratio reached as high as 88.2 in multiplex serology vs 15.4 in the ELISA (Table 1 ). The increased sensitivity of multiplex serology was also reflected by the higher median and mean ratios in both vaccination groups. When we used a ratio >2 and a postvaccination value above cutoff as criteria for E7 vaccination response, multiplex serology identified 11 and 7 responders of 12 patients each in the high- and low-dose groups, respectively, vs only 6 and 4 identified by ELISA. Neither assay showed responders in the placebo group. Weak E7 antibody responses among the prevaccination sera may have been induced as a consequence of the HPV 16 infection underlying the cervical lesion. We concluded that multiplex serology detects weak antibody responses more efficiently than does the GST capture ELISA and thus appears to be more sensitive.



View larger version (19K):
[in this window]
[in a new window]
 
Figure 7. Sensitivity of multiplex serology (top) compared with GST capture ELISA (bottom).

Sera taken from participants of a HPV 16 L1/E7 vaccination trial before ({blacksquare}) and after ({cjs2108}) vaccination were investigated for antibodies against HPV 16 E7. Patients had received a high dose of vaccine (patients 1–12), a low dose of vaccine (patients 13–24), or placebo (patients 25–36). mAU450, milliabsorbance units at 450 nm.


View this table:
[in this window]
[in a new window]
 
Table 1. Sensitivity of multiplex serology and GST capture ELISA.1

application of multiplex serology in epidemiologic studies on cervical cancer with 27 hpv antigens
Sera from 347 women with and 409 women without cervical cancer were analyzed in parallel for antibodies against the E6, E7, and L1 proteins of 9 mucosal HPV types. The high-risk HPV types 16, 18, 31, 33, 35, 45, 52, and 58 are associated with cervical cancer, and HPV type 6b is a low-risk type that is not associated with cervical cancer.

Cervical carcinoma cells mostly contain DNA of only one high-risk HPV type (3), which is also assumed to induce the E6 and E7 antibodies found in the patient serum. We directly compared antibody reactivities of the 756 sera to the homologous E6, E7, and L1 proteins of the 2 most frequent HPV types, 16 and 18 (Fig. 8 ). The majority of positive antibody reactions were directed toward only 1 of the 2 homologous proteins. This finding suggests the absence of substantial antibody cross-reactivity between HPV types 16 and 18 and demonstrates the potential of HPV type–specific antibody detection by multiplex serology.



View larger version (18K):
[in this window]
[in a new window]
 
Figure 8. HPV type–specific detection of antibodies by multiplex serology.

Sera from 756 women with or without cervical cancer were investigated for antibodies against the E6, E7, and L1 proteins of 9 different HPV types in parallel. Shown here are the results for HPV types 16 and 18. Each data point represents the reactions of one serum with the 2 homologous proteins.

Multiplex serology showed a strong association of cervical cancer with antibodies to E6 proteins (Fig. 9 ; data for E7 and L1 antibodies not shown). This association was reported previously for HPV 16 and 18 (5)(6), but is shown here for the first time for 2 less prevalent high-risk HPV types. The Wilcoxon rank-sum test showed differences between cases and controls that were highly significant (P <0.001) for HPV types 16, 52, and 58 and less significant for HPV 18 (P = 0.004). We found no significant difference for HPV types 31 (P = 0.657), 33 (P = 0.077), 35 (P = 0.332), and 45 (P = 0.951) or for the low-risk HPV type 6b (P = 0.511).



View larger version (18K):
[in this window]
[in a new window]
 
Figure 9. Association of antibodies to E6 proteins of 9 HPV types with cervical cancer.

Shown are E6 antibody reactivities in sera from 347 women with cervical cancer (•; left-hand plot in each pair) and from 409 women without cervical cancer ({circ}; right-hand plot in each pair; same study as described in legend to Fig. 8Up ). For each HPV type, the frequency distribution of the individual reactions in cases and controls is shown by separate box plots. Gray boxes encompass the 25th to 75th percentiles, with the line within each box indicating the median MFI value. Whiskers below and above each box indicate the 10th and 90th percentiles, respectively. Individual values outside these boundaries are shown by circles. For groups with few positive antibody reactions, resolution of box plot elements below the 90th percentile is low.


   Discussion
Top
Abstract
Introduction
Materials and Methods
Results
Discussion
References
 
The serologic method presented here combines a well-validated modular expression system for protein antigens and their uniform in situ affinity purification (21)(22) with a high-throughput low-density suspension array. This parallel approach allows large and complex seroepidemiologic studies with several thousand sera and up to 100 antigens. A study with 27 HPV antigens is described here that would hardly be feasible with nonparallel assay formats.

Multiplex serology requires much less time and normalization effort than do conventional serologic assays, such as ELISA. Data acquisition for large epidemiologic studies can be achieved in a few days under almost identical conditions for all samples. Our data for HPV 16 E7 antibodies in the vaccination study indicate that this method has greater sensitivity than GST capture ELISA. Another advantage is the low serum volume required: 2 µL of serum is sufficient to establish antibody profiles for <100 antigens.

The simple expression system for GST fusion proteins and their 1-step purification directly on GC beads offers a substantial workload reduction. In addition, GST-mediated binding selects for fusion proteins with native conformation of at least the GST part, whereas direct coupling of different proteins to beads may lead to grossly varying protein densities (20) and may also affect conformation, particularly of small proteins.

During the development of this multiplex serology method, we identified 2 critical steps: reproducibly high efficiency of GC coupling to the beads and suppression of nonspecific antibody binding to the beads themselves.

With experience, our initial 3-fold or greater differences in GC density on the beads were reduced to interbatch variations <5% (data not shown).

We frequently observed nonspecific binding of human antibodies to beads, even without coupled GC, with background values exceeding 500 MFI in up to 20% of sera from certain collections. Testing several protocols and reagents, we identified a mixture of 3 blocking reagents (polyvinyl alcohol, polyvinylpyrrolidone, and a proprietary reagent, Super ChemiBlock) that, when added to the serum preincubation buffer, suppressed this background sufficiently (median background values <50 MFI with <0.5% of sera giving MFI values >200; Waterboer T, et al. Suppression of nonspecific binding in serological Luminex assays. Manuscript in preparation).

Multiplex serology is robust and versatile. Antibody assays have been developed not only for a total of 54 HPV antigens but also for capsid proteins and large T antigens of 4 polyomaviruses, the main proteins of Lassa and Borna-Disease viruses, and 2 cellular proteins (p53 and p16Ink4a; Michael Pawlita, unpublished data).

In conclusion, we believe that multiplex serology may replace standard ELISA technology, particularly for epidemiologic settings. Applications beyond epidemiology could be the monitoring of infections during pregnancy and the differential diagnosis of infections with similar clinical symptoms.


   Acknowledgments
 
We thank Monika Oppenländer and Ute Koch (DKFZ) for excellent technical assistance; Oliver Pötz, Jochen Schwenk (NMI), and Astrid Gödde [Deutsches Ressourcenzentrum für Genomforschung (RZPD), Heidelberg, Germany] for initial help with the Luminex technology; Lutz Gissmann (DKFZ) for critical reading of the manuscript; Jörg Hoheisel (DKFZ) for providing laboratory space; Ignacio G. Bravo (DKFZ) for help with statistical analyses; and T. Rajkumar (Cancer Institute, Women India Association, Chennai, India) and D. Hammouda (Registre des Tumeurs d’Alger, Institut National de Sante Publique, Algiers, Algeria) for providing serum samples. Plasmids for HPV types 31 and 35 were a gift from A. Lörincz (Digene Corp., Gaithersburg, MD), plasmids for HPV 33 were a gift from G. Orth (Institut Pasteur, Paris, France), plasmids for HPV 45 were a gift from K. Shah (Johns Hopkins Bloomberg School of Public Health, Baltimore, MD), plasmids for HPV 52 were a gift from W. Lancaster (Wayne State University, Detroit, MI), and plasmids for HPV 58 were a gift from T. Matsukura (National Institute of Health, Tokyo, Japan). We gratefully acknowledge the financial support of the European Union (QLK2-CT-2002-01179) and the Roland Ernst-Stiftung (Essen, Germany).


   Footnotes
 
1 Current address: Joint DKFZ-EMBL Chemical Biology Core Facility, European Molecular Biology Laboratory (EMBL), Meyerhofstrasse 1, 69117 Heidelberg, Germany.

2 Nonstandard abbreviations: HPV, human papillomavirus; GST, glutathione S-transferase; GC, glutathione–casein; PBS, phosphate-buffered saline; MFI, median fluorescence intensity; and PE, R-phycoerythrin.


   References
Top
Abstract
Introduction
Materials and Methods
Results
Discussion
References
 

  1. De Villiers EM, Fauquet C, Broker TR, Bernard HU, Zur Hausen H. Classification of papillomaviruses. Virology 2004;324:17-27.[CrossRef][Web of Science][Medline] [Order article via Infotrieve]
  2. zur Hausen H. Papillomavirus infections–a major cause of human cancers. Biochim Biophys Acta 1996;1288:F55-F78.[Medline] [Order article via Infotrieve]
  3. Munoz N, Bosch FX, de Sanjose S, Herrero R, Castellsague X, Shah KV, et al. Epidemiologic classification of human papillomavirus types associated with cervical cancer. N Engl J Med 2003;348:518-527.[Abstract/Free Full Text]
  4. Dillner J. The serological response to papillomaviruses. Semin Cancer Biol 1999;9:423-430.[CrossRef][Web of Science][Medline] [Order article via Infotrieve]
  5. Zumbach K, Kisseljov F, Sacharova O, Shaichaev G, Semjonova L, Pavlova L, et al. Antibodies against oncoproteins E6 and E7 of human papillomavirus types 16 and 18 in cervical-carcinoma patients from Russia. Int J Cancer 2000;85:313-318.[CrossRef][Web of Science][Medline] [Order article via Infotrieve]
  6. Meschede W, Zumbach K, Braspenning J, Scheffner M, Benitez-Bribiesca L, Luande J, et al. Antibodies against early proteins of human papillomaviruses as diagnostic markers for invasive cervical cancer. J Clin Microbiol 1998;36:475-480.[Abstract/Free Full Text]
  7. Templin MF, Stoll D, Schwenk JM, Potz O, Kramer S, Joos TO. Protein microarrays: promising tools for proteomic research. Proteomics 2003;3:2155-2166.[CrossRef][Web of Science][Medline] [Order article via Infotrieve]
  8. Templin MF, Stoll D, Bachmann J, Joos TO. Protein microarrays and multiplexed sandwich immunoassays: what beats the beads?. Comb Chem High Throughput Screen 2004;7:223-229.[Web of Science][Medline] [Order article via Infotrieve]
  9. Dunbar SA, Vander Zee CA, Oliver KG, Karem KL, Jacobson JW. Quantitative, multiplexed detection of bacterial pathogens: DNA and protein applications of the Luminex LabMAP system. J Microbiol Methods 2003;53:245-252.[CrossRef][Web of Science][Medline] [Order article via Infotrieve]
  10. Prabhakar U, Eirikis E, Davis HM. Simultaneous quantification of proinflammatory cytokines in human plasma using the LabMAP assay. J Immunol Methods 2002;260:207-218.[CrossRef][Web of Science][Medline] [Order article via Infotrieve]
  11. Lee SH, Walker DR, Cregan PB, Boerma HR. Comparison of four flow cytometric SNP detection assays and their use in plant improvement. Theor Appl Genet 2004;110:167-174.[CrossRef][Web of Science][Medline] [Order article via Infotrieve]
  12. Seideman J, Peritt D. A novel monoclonal antibody screening method using the Luminex-100 microsphere system. J Immunol Methods 2002;267:165-171.[CrossRef][Web of Science][Medline] [Order article via Infotrieve]
  13. Jones LP, Zheng HQ, Karron RA, Peret TC, Tsou C, Anderson LJ. Multiplex assay for detection of strain-specific antibodies against the two variable regions of the G protein of respiratory syncytial virus. Clin Diagn Lab Immunol 2002;9:633-638.[Abstract/Free Full Text]
  14. Opalka D, Lachman CE, MacMullen SA, Jansen KU, Smith JF, Chirmule N, et al. Simultaneous quantitation of antibodies to neutralizing epitopes on virus-like particles for human papillomavirus types 6, 11, 16, and 18 by a multiplexed luminex assay. Clin Diagn Lab Immunol 2003;10:108-115.[Abstract/Free Full Text]
  15. Bellisario R, Colinas RJ, Pass KA. Simultaneous measurement of antibodies to three HIV-1 antigens in newborn dried blood-spot specimens using a multiplexed microsphere-based immunoassay. Early Hum Dev 2001;64:21-25.[CrossRef][Web of Science][Medline] [Order article via Infotrieve]
  16. Wong SJ, Demarest VL, Boyle RH, Wang T, Ledizet M, Kar K, et al. Detection of human anti-flavivirus antibodies with a west nile virus recombinant antigen microsphere immunoassay. J Clin Microbiol 2004;42:65-72.[Abstract/Free Full Text]
  17. Biagini RE, Sammons DL, Smith JP, MacKenzie BA, Striley CA, Semenova V, et al. Comparison of a multiplexed fluorescent covalent microsphere immunoassay and an enzyme-linked immunosorbent assay for measurement of human immunoglobulin G antibodies to anthrax toxins. Clin Diagn Lab Immunol 2004;11:50-55.[Abstract/Free Full Text]
  18. Pickering JW, Martins TB, Schroder MC, Hill HR. Comparison of a multiplex flow cytometric assay with enzyme-linked immunosorbent assay for quantitation of antibodies to tetanus, diphtheria, and Haemophilus influenzae type b. Clin Diagn Lab Immunol 2002;9:872-876.[Abstract/Free Full Text]
  19. Gilburd B, Abu-Shakra M, Shoenfeld Y, Giordano A, Bocci EB, delle Monache F, et al. Autoantibodies profile in the sera of patients with Sjogren’s syndrome: the ANA evaluation—a homogeneous, multiplexed system. Clin Dev Immunol 2004;11:53-56.[CrossRef][Medline] [Order article via Infotrieve]
  20. Dasso J, Lee J, Bach H, Mage RG. A comparison of ELISA and flow microsphere-based assays for quantification of immunoglobulins. J Immunol Methods 2002;263:23-33.[CrossRef][Web of Science][Medline] [Order article via Infotrieve]
  21. Sehr P, Zumbach K, Pawlita M. A generic capture ELISA for recombinant proteins fused to glutathione S-transferase: validation for HPV serology. J Immunol Methods 2001;253:153-162.[CrossRef][Web of Science][Medline] [Order article via Infotrieve]
  22. Sehr P, Muller M, Hopfl R, Widschwendter A, Pawlita M. HPV antibody detection by ELISA with capsid protein L1 fused to glutathione S-transferase. J Virol Methods 2002;106:61-70.[CrossRef][Web of Science][Medline] [Order article via Infotrieve]
  23. Staros JV, Wright RW, Swingle DM. Enhancement by N-hydroxysulfosuccinimide of water-soluble carbodiimide-mediated coupling reactions. Anal Biochem 1986;156:220-222.[CrossRef][Web of Science][Medline] [Order article via Infotrieve]
  24. Franceschi S, Rajkumar T, Vaccarella S, Gajalakshmi V, Sharmila A, Snijders PJ, et al. Human papillomavirus and risk factors for cervical cancer in Chennai, India: a case-control study. Int J Cancer 2003;107:127-133.[CrossRef][Web of Science][Medline] [Order article via Infotrieve]
  25. Hammouda D, Munoz N, Herrero R, Arslan A, Bouhadef A, Oublil M, et al. Cervical carcinoma in Algiers, Algeria: human papillomavirus and lifestyle risk factors. Int J Cancer 2005;113:483-489.[CrossRef][Web of Science][Medline] [Order article via Infotrieve]
  26. MacArthur H, Walter G. Monoclonal antibodies specific for the carboxy terminus of simian virus 40 large T antigen. J Virol 1984;52:483-491.[Abstract/Free Full Text]
  27. Roederer M. PE conjugation of antibodies. http://www.drmr.com/abcon/PE.html (accessed March 2005)..
  28. Friguet B, Chaffotte AF, Djavadi-Ohaniance L, Goldberg ME. Measurements of the true affinity constant in solution of antigen-antibody complexes by enzyme-linked immunosorbent assay. J Immunol Methods 1985;77:305-319.[CrossRef][Web of Science][Medline] [Order article via Infotrieve]
  29. . Clinical and Laboratory Standards Institute. User evaluation of precision performance of clinical chemistry devices. Tentative guideline EP5-T June 1984 CLSI Wayne PA. .
  30. Quantify agreement with kappa. http://www.graphpad.com/quickcalcs/kappa1.cfm (accessed March 2005)..



The following articles in journals at HighWire Press have cited this article:


Home page
Nucleic Acids ResHome page
U. Schaffer, A. Schlosser, K. M. Muller, A. Schafer, N. Katava, R. Baumeister, and E. Schulze
SnAvi - a new tandem tag for high-affinity protein-complex purification
Nucleic Acids Res., January 4, 2010; (2010) gkp1178v1.
[Abstract] [Full Text] [PDF]


Home page
J. Virol.Home page
T. Senger, M. R. Becker, L. Schadlich, T. Waterboer, and L. Gissmann
Identification of B-Cell Epitopes on Virus-Like Particles of Cutaneous Alpha-Human Papillomaviruses
J. Virol., December 15, 2009; 83(24): 12692 - 12701.
[Abstract] [Full Text] [PDF]


Home page
J. Gen. Virol.Home page
T. Waterboer, R. Neale, K. M. Michael, P. Sehr, M. N. C. de Koning, S. J. Weissenborn, F. Sampogna, D. Abeni, A. C. Green, J. N. Bouwes Bavinck, et al.
Antibody responses to 26 skin human papillomavirus types in the Netherlands, Italy and Australia
J. Gen. Virol., August 1, 2009; 90(8): 1986 - 1998.
[Abstract] [Full Text] [PDF]


Home page
Cancer Res.Home page
L. Gao, A. Michel, M. N. Weck, V. Arndt, M. Pawlita, and H. Brenner
Helicobacter pylori Infection and Gastric Cancer Risk: Evaluation of 15 H. pylori Proteins Determined by Novel Multiplex Serology
Cancer Res., August 1, 2009; 69(15): 6164 - 6170.
[Abstract] [Full Text] [PDF]


Home page
J. Gen. Virol.Home page
S. Syrjanen, T. Waterboer, M. Sarkola, K. Michael, M. Rintala, K. Syrjanen, S. Grenman, and M. Pawlita
Dynamics of human papillomavirus serology in women followed up for 36 months after pregnancy
J. Gen. Virol., June 1, 2009; 90(6): 1515 - 1526.
[Abstract] [Full Text] [PDF]


Home page
Cancer Res.Home page
L. Gao, M. N. Weck, A. Michel, M. Pawlita, and H. Brenner
Association between Chronic Atrophic Gastritis and Serum Antibodies to 15 Helicobacter pylori Proteins Measured by Multiplex Serology
Cancer Res., April 1, 2009; 69(7): 2973 - 2980.
[Abstract] [Full Text] [PDF]


Home page
Cancer Epidemiol. Biomarkers Prev.Home page
B. Dondog, G. M. Clifford, S. Vaccarella, T. Waterboer, D. Unurjargal, D. Avirmed, S. Enkhtuya, F. Kommoss, N. Wentzensen, P. J.F. Snijders, et al.
Human Papillomavirus Infection in Ulaanbaatar, Mongolia: A Population-Based Study
Cancer Epidemiol. Biomarkers Prev., July 1, 2008; 17(7): 1731 - 1738.
[Abstract] [Full Text] [PDF]


Home page
J. Gen. Virol.Home page
R. Z. Rizk, N. D. Christensen, K. M. Michael, M. Muller, P. Sehr, T. Waterboer, and M. Pawlita
Reactivity pattern of 92 monoclonal antibodies with 15 human papillomavirus types
J. Gen. Virol., January 1, 2008; 89(1): 117 - 129.
[Abstract] [Full Text] [PDF]


Home page
Cancer Epidemiol. Biomarkers Prev.Home page
K. Andersson, T. Waterboer, R. Kirnbauer, K. Slupetzky, T. Iftner, E.-M. de Villiers, O. Forslund, M. Pawlita, and J. Dillner
Seroreactivity to Cutaneous Human Papillomaviruses among Patients with Nonmelanoma Skin Cancer or Benign Skin Lesions
Cancer Epidemiol. Biomarkers Prev., January 1, 2008; 17(1): 189 - 195.
[Abstract] [Full Text] [PDF]


Home page
JCOHome page
D. A.M. Heideman, T. Waterboer, M. Pawlita, P. Delis-van Diemen, I. Nindl, J. A. Leijte, J. M.G. Bonfrer, S. Horenblas, C. J.L.M. Meijer, and P. J.F. Snijders
Human Papillomavirus-16 Is the Predominant Type Etiologically Involved in Penile Squamous Cell Carcinoma
J. Clin. Oncol., October 10, 2007; 25(29): 4550 - 4556.
[Abstract] [Full Text] [PDF]


Home page
Cancer Epidemiol. Biomarkers Prev.Home page
G. M. Clifford, H.-R. Shin, J.-K. Oh, T. Waterboer, Y.-H. Ju, S. Vaccarella, W. Quint, M. Pawlita, and S. Franceschi
Serologic Response to Oncogenic Human Papillomavirus Types in Male and Female University Students in Busan, South Korea
Cancer Epidemiol. Biomarkers Prev., September 1, 2007; 16(9): 1874 - 1879.
[Abstract] [Full Text] [PDF]


Home page
CVIHome page
A. J. Johnson, R. C. Cheshier, G. Cosentino, H. P. Masri, V. Mock, R. Oesterle, R. S. Lanciotti, D. A. Martin, A. J. Panella, O. Kosoy, et al.
Validation of a Microsphere-Based Immunoassay for Detection of Anti-West Nile Virus and Anti-St. Louis Encephalitis Virus Immunoglobulin M Antibodies
Clin. Vaccine Immunol., September 1, 2007; 14(9): 1084 - 1093.
[Abstract] [Full Text] [PDF]


Home page
J Biomol ScreenHome page
P. Sehr, M. Pawlita, and J. Lewis
Evaluation of Different Glutathione S-Transferase-Tagged Protein Captures for Screening E6/E6AP Interaction Inhibitors Using AlphaScreen(R)
J Biomol Screen, June 1, 2007; 12(4): 560 - 567.
[Abstract] [PDF]


Home page
JNCI J Natl Cancer InstHome page
M. R. Karagas, H. H. Nelson, P. Sehr, T. Waterboer, T. A. Stukel, A. Andrew, A. C. Green, J. N. Bouwes Bavinck, A. Perry, S. Spencer, et al.
Human papillomavirus infection and incidence of squamous cell and basal cell carcinomas of the skin.
J Natl Cancer Inst, March 15, 2006; 98(6): 389 - 395.
[Abstract] [Full Text] [PDF]


Home page
J. Clin. Microbiol.Home page
M. Schmitt, I. G. Bravo, P. J. F. Snijders, L. Gissmann, M. Pawlita, and T. Waterboer
Bead-Based Multiplex Genotyping of Human Papillomaviruses
J. Clin. Microbiol., February 1, 2006; 44(2): 504 - 512.
[Abstract] [Full Text] [PDF]


Home page
Clin. Chem.Home page
J. Dillner
Toward "Serolomics": Papillomavirus Serology Is Taking a Technologic Lead in High-Throughput Multiplexed Antibody Analysis
Clin. Chem., October 1, 2005; 51(10): 1768 - 1769.
[Full Text] [PDF]


This Article
Right arrow Abstract Freely available
Right arrow Full Text (PDF)
Right arrow All Versions of this Article:
clinchem.2005.052381v1
51/10/1845    most recent
Right arrow Alert me when this article is cited
Right arrow Alert me if a correction is posted
Right arrow Citation Map
Services
Right arrow Email this article to a friend
Right arrow Similar articles in this journal
Right arrow Similar articles in Web of Science
Right arrow Similar articles in PubMed
Right arrow Alert me to new issues of the journal
Right arrow Download to citation manager
Right arrow reprints & permissions
Citing Articles
Right arrow Citing Articles via HighWire
Right arrow Citing Articles via Web of Science (34)
Right arrow Citing Articles via Google Scholar
Google Scholar
Right arrow Articles by Waterboer, T.
Right arrow Articles by Pawlita, M.
Right arrow Search for Related Content
PubMed
Right arrow PubMed Citation
Right arrow Articles by Waterboer, T.
Right arrow Articles by Pawlita, M.
Related Collections
Right arrow Automation and Analytical Techniques


HOME HELP FEEDBACK SUBSCRIPTIONS ARCHIVE SEARCH TABLE OF CONTENTS