|
|
||||||||
Evidence-Based Laboratory Medicine and Test Utilization |
Departments of1 Laboratory Medicine & Immunology and 2 Rheumatology, University Hospital Leuven, Belgium.
aAddress correspondence to this author at: Department of Laboratory Medicine, Immunology, University Hospital Leuven, Herestraat 49, B-3000 Leuven, Belgium. Fax 0032-16-347931; e-mail xavier.bossuyt{at}uz.kuleuven.ac.be
| Abstract |
|---|
|
|
|---|
Methods: ACPA were determined in 298 serum samples using 6 commercially available ACPA assays. One hundred two samples were from RA patients, including patients with early and established RA, and 196 were from controls, including patients with psoriatic arthritis, connective tissue diseases, organ-specific autoimmune diseases, and a group of consecutive patients for whom a rheumatologist ordered anticyclic citrullinated peptide (CCP) antibodies. The ELISA reagent sets under study were Citrullinated Protein Antibodies (Genesis), Anti-MCV (Orgentec), Immunoscan RA (Euro-Diagnostica), Anti-CCP IgG ELISA (Euroimmun), EliATM CCP (Phadia), and Quanta LiteTM CCP3 IgG ELISA (Inova). Technical performance (imprecision, linearity, correlation, and agreement) and diagnostic accuracy (sensitivity and specificity) were compared.
Results: Variable technical performance was noted among the different ACPA assays, with some assays displaying poor reproducibility and bad linearity. ACPA results were well correlated among assays with the same antigen specificity, but the numerical values reported for each assay differed widely. Using cutoff values proposed by the manufacturer, diagnostic sensitivities ranged between 69.6% and 77.5% and specificities between 87.8% and 96.4%. The areas under the ROC curves were comparable among the different assays.
Conclusions: Overall diagnostic performance of ACPA assays is comparable among the different assays, but standardization is needed. For some assays, analytical characteristics could be improved.
| Introduction |
|---|
|
|
|---|
0.8% of the world population. RA is characterized by an inflammation of synovial joints, which often leads to progressive joint destruction and disability (1). This poor prognosis has led to an emphasis on early treatment (2). Unfortunately, early diagnosis is difficult. Autoantibody formation is a common manifestation of RA (3). The best-known antibody is rheumatoid factor, the presence of which is one of the American College of Rheumatology (ACR) classification criteria for RA (4). Despite its lack of specificity, rheumatoid factor is widely used as a diagnostic marker (5).
A variety of other antibodies specific for RA have been discovered and reported to be of diagnostic value. In 1964, Nienhuis et al. described the antiperinuclear factor (6). Fifteen years later, Young et al. detected antikeratin antibodies (7). Those antibodies were shown to belong to the same family of autoantibodies directed to filaggrin and/or its precursor profillagrin (8)(9). This protein is found in epithelial cells of mammals. Epitopes recognized by the antifilaggrin antibodies have been shown to contain citrullinated residues, converted from arginine to citrulline by the enzyme peptidylarginine deiminase (10)(11). An ELISA, based on citrullinated recombinant rat filaggrin, has been developed to detect antifillagrin antibodies (12). Sensitivity was enhanced substantially without loss of specificity by the use of synthetic cyclic peptides derived from the sequence of human filaggrin with a high content of citrullin [cyclic citrullinated peptides (CCP)] as antigen (13). The first available CCP (1st generation) was further optimized by screening of dedicated peptide libraries (2nd generation).
Another member of the family of antibodies directed to citrullinated proteins are anti-Sa antibodies. The Sa antigen is present in RA synovial tissue and has been identified as citrullinated vimentin (14). One isoform of vimentin, mutated citrullinated vimentin, has been shown to be sensitive and highly specific for RA by Orgentec (15).
In this study we compared the diagnostic accuracies of 6 ELISAs for detecting anticitrullinated protein/peptide antibodies (ACPA): 1 anticitrullinated rat filaggrin antibody assay, 1 antimutated citrullinated vimentin antibody assay, and 4 anti-CCP antibody assays.
| Materials and Methods |
|---|
|
|
|---|
All serum samples were stored at 20 °C until they were assayed. The study was approved by the local Ethics Committee. Informed consent, however, was not requested because the anti-CCP assay is a routine test performed in our laboratory.
determination of acpa
Six commercially available ACPA ELISAs were evaluated (for details, see Table 1 in the online Data Supplement). The Genesis, Orgentec, Euro-Diagnostica, Euroimmun, and Inova assays were performed on an automated ELISA instrument, the BEP III (Dade-Behring). Serum dilutions were made on the PhDTM System (Bio-Rad). The Phadia assay was performed on the ImmunoCap 100 instrument (Phadia). The procedures were conducted according to the manufacturers recommendations. All measurements were performed by experienced operators who were blinded to the results of other tests.
statistical analysis
ROC analysis, Spearman rank correlations, PassingBablok analysis, and AltmanBland analysis were performed with Analyze-ItTM for Microsoft Excel (version 1.62). The areas under the ROC curve were compared by use of the nonparametric method of DeLong et al. (22). This method takes into account possible correlations between the areas under curve, given that the same patients underwent all tests. A limitation of the area under the ROC curve is that it is a global measure of the diagnostic performance of a test, irrespective of the shape of the ROC curve (23)(24).
In preparing this report, the STARD guidelines (25) were taken into account wherever possible.
| Results |
|---|
|
|
|---|
Linearity.
We determined assay linearity by diluting serum samples containing ACPA with increasing amounts of a serum sample that did not contain ACPA. The ratio of the volume of the sample containing ACPA to the total volume was plotted against the measured ACPA value (see Fig. 1 in the online Data Supplement). The regression line was calculated using the PassingBablok method. The Cusum test revealed significant deviation from linearity (0.05 > P <0.1) for Genesis (samples F and G), Euro-Diagnostica (sample G), and Inova (sample F). Although the Cusum test did not show significant deviation from linearity, the regression line visually looked problematic for Euro-Diagnostica and Orgentec at higher values. The best results were achieved with Euroimmun.
Method comparison.
The median and range of ACPA titers in RA samples varied substantially among the methods (see Table 1 in the online Data Supplement). For evaluating the degree of association among the different assays, a nonparametric approach was chosen (Spearman rank correlation coefficients; see Table 3 in the online Data Supplement). Low correlation coefficients were observed when comparing with Inova and Genesis. The best correlations were found between assays using the 2nd generation CCP antigen (Euro-Diagnostica, Euroimmun, and Phadia) with correlation coefficients ranging between 0.90 and 0.95. For evaluating the degree of agreement, PassingBablok regression analysis was used to determine proportional and constant bias via, respectively, slopes and intercepts of the regression lines of pairwise assay comparisons (see Table 3 in the online Data Supplement). Poor concordance was observed between assays in these comparisons with large deviations away from the target values of 1.00 for slopes and 0.00 for intercepts. Additionally, AltmanBland analysis (plotting the means of the results of two methods on the x-axis and the differences between those methods on the y-axis) was used to assess the mean differences between methods (bias; see Table 3 in the online Data Supplement). Visual inspection of the AltmanBland plots revealed titer-dependent differences between assays (data not shown).
diagnostic performance
Distributions of ACPA concentrations in patients with RA and in control patients for the different assays are shown in Fig. 1
, and the number of positive results for ACPA in RA patients and in the control groups are displayed in Table 1
. Using the cutoff values proposed by the manufacturers 71 (69.6%), 76 (74.5%), 78 (76.5%), 74 (72.5%), 79 (77.5%), and 79 (77.5%) of 102 sera from RA patients and 12 (6.1%), 19 (9.7%), 9 (4.6%), 7 (3.6%), 8 (4.1%), and 24 (12.2%) of 196 control sera tested positive for Genesis, Orgentec, Euro-Diagnostica, Euroimmun, Phadia, and Inova methods, respectively. Sixty-four (62.7%) of 102 sera from RA patients (28 of 40 established RA patients, 26 of 45 early RA patients, 10 of 16 consecutively recruited RA patients, and 0 of 1 patient with myasthenia gravis and RA) tested positive with all the assays used. Forty-one (20.9%) of 196 control sera (6 of 32 consecutively recruited non-RA patients, 5 of 27 psoriatic arthritis patients, 27 of 98 patients with connective tissue diseases, and 3 of 39 patients with specific autoantibodies) showed a positive reaction with 1 or more assays. Five (2.6%) control sera (3 psoriatic arthritis, 1 Sjögren syndrome, and 1 scleroderma) showed a positive reaction with all assays. The patients with psoriatic arthritis who tested positive with all assays fulfilled the ACR criteria for RA. A detailed description of the reactivity of the control sera with the various assays is given in Text 2 in the online Data Supplement.
|
|
Sensitivity and specificity were calculated with respect to the clinical diagnosis for each method (Table 2
). The areas under the ROC curve were 0.851, 0.866, 0.863, 0.855, 0.879, and 0.884 for Genesis, Orgentec, Euro-Diagnostica, Euroimmun, Phadia and Inova methods, respectively. No significant differences in areas under the ROC curve were found among the different assay methods (P = 0.086).
|
Optimum cutoff values based on the highest sum of sensitivity and specificity were estimated and are shown in Table 2
. According to our analysis the cutoff value for the Inova assay should be higher than that suggested by the manufacturer (79.3 units vs 20 units). Diagnostic performance was calculated for each assay at these optimum cutoff values (Table 2
).
| Discussion |
|---|
|
|
|---|
Variable technical performance was found. For example, the within-run imprecision and linearity was problematic for Euro-Diagnostica. The between-run imprecision could not be assessed because we performed only a limited number of runs. Low correlation was observed between the methods using different antigens, but there was good correlation between the 3 2nd generation anti-CCP assays. Nevertheless, calibration differences between those methods preclude use of their results interchangeably. The international scientific community and the commercial private sector should undertake efforts to harmonize the assays. Currently, international reference sera of defined ACPA content and specificity are not available. Each manufacturer uses its own calibrator to define arbitrary units. Differences in reactivity between ACPA calibrators and/or differences in the citrullinated epitopes can be identified as sources of poor comparability among methods. Complete harmonization of assays is only possible if the same antigen is used.
A cautionary note is warranted with respect to our study design of the evaluation of the diagnostic performance. The majority of patients were selected for the study based on their known clinical diagnosis. Although such a selection scheme carries the advantage that the diagnoses are firmly established, it has the disadvantage that the study population does not reflect everyday rheumatology practice. We attempted to overcome this disadvantage by including a limited group of consecutive patients for whom a rheumatologist ordered anti-CCP antibodies. However, a potential problem in this latter group of patients is that the clinicians were aware of the anti-CCP test results at the time they established the diagnosis. For the different assays, the observed sensitivities in the established and early RA groups varied between 75.0% and 82.5% and between 64.4% and 73.3%, respectively, in agreement with most previously published studies (27)(28). Likewise, the specificities we observed varied between 87.8% and 96.4% and are very close to previously reported values (27)(29). The areas under the ROC curve could not be shown to be statistically different among the assays evaluated, suggesting that the diagnostic performance of the various ACPA assays was comparable. For the Inova assay, an adaptation of the cutoff value may be desirable. Caution is warranted with respect to the calculated predictive values because predictive values depend on the prevalence of RA in the population examined.
The differential diagnosis between RA in patients with psoriasis and psoriatic arthritis may be difficult (30). Using the ACR criteria for RA is not helpful, because many patients with psoriatic arthritis fulfill those criteria (31). We found that 3 of 27 patients (11.1%) with psoriatic arthritis tested positive with all ACPA assays. The question is whether these positive patients truly have psoriatic arthritis or whether they represent patients who actually have RA and psoriasis. Although they fulfilled the ACR criteria for RA, they were considered to have psoriatic arthritis and not RA because of the typical distribution of the involved joints and the presence of psoriatic skin lesions. Our results are in concordance with a study from Vander Cruyssen et al. (32), who found anti-CCP antibodies in 7.8% of 192 psoriatic arthritis patients.
Other studies reported the presence of anti-CCP antibodies in 11 of 201 (5.5%) sera obtained from patients with SLE and in 10 of 134 (7.5%) sera obtained from patients with Sjögren syndrome (33)(34). We found that the prevalence of ACPA in patients with SLE, Sjögren syndrome, and scleroderma varies depending on the assay. The prevalences of ACPA across assays in 41 SLE patients varied between 2.4% and 17.1%. None of the 41 patients was positive with all ACPA assays. The prevalences of ACPA in 26 patients with Sjögren syndrome varied between 3.8% and 19.2%. One of the 26 patients (3.8%) was positive with all ACPA assays. The prevalences of ACPA in 18 scleroderma patients varied between 5.6% and 33.3%. One of the 18 patients (5.6%) was positive with all ACPA assays.
Several studies evaluated the effect of treatment on anti-CCP antibody titers in RA patients. Alessandri et al. (35) found that anti-CCP titers decreased after 6 months of infliximab treatment. Bobbio-Pallavicini et al. (36) reported a significant decrease in anti-CCP antibody titer at 30 weeks during infliximab and methotrexate therapy, but titers returned to baseline thereafter. Chen et al. (37) found a significant decrease of anti-CCP serum concentrations after 3 months of etanercept treatment. Mikuls et al. (38) reported a significant decrease of anti-CCP titers over the follow-up period among RA patients of whom a majority received active therapy with disease-modifying antirheumatic drugs. It should be noted that if the anti-CCP determination is repeated over time, technical aspects are critically important. None of the above-mentioned studies took into account the technical performance of the anti-CCP assay used. The validity of studies evaluating the effect of treatment on anti-CCP titers using nonlinear and nonprecise tests should be questioned.
In conclusion, overall diagnostic performance of ACPA assays is comparable among the different assays, but standardization is needed. For some assays, analytical characteristics could be improved.
| Acknowledgments |
|---|
| Footnotes |
|---|
| References |
|---|
|
|
|---|
The following articles in journals at HighWire Press have cited this article:
![]() |
X. Bossuyt, C. Louche, and A. Wiik Standardisation in clinical laboratory medicine: an ethical reflection Ann Rheum Dis, August 1, 2008; 67(8): 1061 - 1063. [Full Text] [PDF] |
||||
![]() |
K Kitahara, K Takagi, Y Kusunoki, S Nishio, T Nozaki, H Inomata, M Takei, S Sawada, and S Kawai Clinical value of second- and third-generation assays of anti-cyclic citrullinated peptide antibodies in rheumatoid arthritis Ann Rheum Dis, July 1, 2008; 67(7): 1059 - 1060. [Full Text] [PDF] |
||||
![]() |
B Vander Cruyssen, L Nogueira, J Van Praet, D Deforce, D Elewaut, G Serre, and F De Keyser Do all anti-citrullinated protein/peptide antibody tests measure the same? Evaluation of discrepancy between anti-citrullinated protein/peptide antibody tests in patients with and without rheumatoid arthritis Ann Rheum Dis, April 1, 2008; 67(4): 542 - 546. [Abstract] [Full Text] [PDF] |
||||
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |