| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
|
|
|
|||||||
|
|||||||||
Evidence-based Laboratory Medicine and Test Utilization |
1 NOKLUS, Norwegian Centre for Quality Improvement of Primary Care Laboratories, Section for General Practice, Department of Public Health and Primary Health Care, University of Bergen, Bergen, Norway.
2 Laboratori Clínic Bon Pastor Barcelona and the Analytical Quality Commission of the Spanish Society of Clinical Biochemistry and Molecular Pathology (SEQC), Barcelona, Spain.
3 Department of Clinical Chemistry, University of Szeged, Albert Szent-Gyorgyi Medical and Pharmaceutical Centre, Szeged, Hungary.
4 Department of Clinical Chemistry, St. Elisabeth Hospital, Tilburg, The Netherlands.
5 The Department of General Practice, The University of Adelaide, Adelaide, Australia.
6 EQUALIS AB, Uppsala, Sweden.
7 Faculty of Natural Sciences, University of Pretoria, Pretoria, South Africa.
aAddress correspondence to this author at: Department of Medicine, Stavanger University Hospital, PO Box 8100, 4068 Stavanger, Norway. Fax 47-51-519-9-06; e-mail svskeie{at}online.no.
 |
Abstract |
|---|
 Top Abstract IntroductionMaterials and MethodsResultsDiscussionReferences |
|---|
Methods: GPs recruited from 7 countries received questionnaires requesting interpretation of changes in a potentially diagnostic capillary BG result and an HbA1c value obtained during monitoring of a patient with type 2 DM. GPs were asked to estimate clinically significant differences between 2 consecutive laboratory results [critical difference (CD)/reference change value] for both BG and HbA1c. The CDs reported by GPs were used to calculate the analytical variation (CVa), which was taken as the quality specification for analytical imprecision. Participants received national benchmarking feedback reports after the survey.
Results: The study included responses from 2538 GPs. CDs in BG results showed the same pattern and were comparable among countries. Calculated median CVa values would be possible to attain at 80% confidence but not at the conventional 95% confidence. For HbA1c, the same pattern was shown across countries, but with lower changes considered true when HbA1c increased than when it decreased. Despite the consistent pattern, variations among GPs were considerable in all countries.
Conclusions: Assessments of CDs for BG and HbA1c were similar internationally, and quality specifications for these analytes based on clinicians’ opinions are therefore interchangeable among countries. International P-EQAS may contribute to a more rational use of laboratory services and clinical guidelines.
|
Introduction |
|---|
|
TopAbstractIntroductionMaterials and MethodsResultsDiscussionReferences |
|---|
In the Norwegian Centre for Quality Improvement of Primary Care Laboratories (NOKLUS), a model for distributing case histories together with regular quality assessment material has been developed and is used systematically as an educational postanalytical external quality assessment survey (P-EQAS). General practitioners (GPs) are asked to respond to a few questions assessing the interpretation of test results in an important and recognizable clinical situation. This approach is used to put external quality assessment into clinical context and to emphasize the importance of analytical quality for medical decision making. NOKLUS has used this approach among Norwegian GPs and in DM patients(4)(5)(6).
To our knowledge, studies comparing the interpretation of laboratory tests in diabetes care in different countries have not been performed. Our study was designed to assess the clinical judgment of practicing GPs in different countries concerning BG and HbA1c in the diagnosis and monitoring of DM. We also aimed to investigate whether our model, as a tool for postanalytical quality assessment, is feasible in other countries.
|
Materials and Methods |
|---|
|
TopAbstractIntroductionMaterials and MethodsResultsDiscussionReferences |
|---|
). No reminders were sent after the GPs received the questionnaire. The questionnaire included 2 short case histories describing typical scenarios in general practice/family medicine. The first case history described a situation related to capillary measurement of BG and the diagnosis of DM, investigating the testing–retesting strategy of GPs. The second case history described a typical type 2 DM patient with a certain HbA1c value at a follow-up visit and investigated the monitoring practices of GPs. Before international distribution, both case histories were reviewed by Norwegian GPs and diabetes specialists with skills in survey methodology. The cases were based on real patient encounters, and care was taken to present them in a form familiar to GPs. The Norwegian questionnaire was translated into English and then into the appropriate native languages. The translated questionnaires were distributed to practicing primary healthcare GPs recruited in each country. Responses from GPs were collected by the national study coordinators, and individual responses were coded and sent anonymously to NOKLUS. Analysis of data was not performed for countries with fewer than 50 responses.
|
The doctors were asked to state higher and lower BG or HbA1c values denoting a true change compared with a preset baseline value, so that both clinical improvement and deterioration would seem plausible to the GPs. However, participants in Sweden and a majority of Norwegian participants carried out the baseline HbA1c measurement themselves in their own office laboratories, using regular EQAS material sent with the questionnaire. The mean baseline HbA1c value was 9.0% in Norway and 8.6% in Sweden, whereas a preset value of 9.1% was used for the other countries. The study was performed in 2002 in all countries but Norway and Sweden, where the original studies had been performed earlier (see below). Methodologic differences among countries are described in the following country-specific sections.
The change between 2 consecutive laboratory results, stated by the GPs, is denoted the critical difference (CD) and is similar to the reference change value(7). The CD is defined as the minimum difference needed between 2 consecutive test results to be certain (with a given degree of confidence) that the 2 results are truly different and not simply a result of analytical imprecision (CVa) or intraindividual biological variation (CVi). The formula for CD is: CD = z-value x 
x 
, with z-values for 1-sided tests and a given probability(8). Random bias is here included in the CVa, which then also will comprise long-term variations. Thus, with CVi known from the literature and the CD of BG or HbA1c stated by doctors, the analytical imprecision (CVa) needed can be calculated. We rearranged the formula to calculate the CVa as an estimate of the quality goal for analytical imprecision, assuming the bias component to be 0(9). Calculations of CVa were performed with a CVi of 5% for BG(10)(11)(12) and 4% for HbA1c(13), with z-values corresponding to 80% (z = 0.84) and 95% (z = 1.64) levels of probability. The 80% probability value was included based on the judgment that the semantics of the questionnaire corresponded better with an 80% probability than the 95% probability most frequently used.
All participants received a feedback report after the survey. The results for each GP were benchmarked in comparison with national results. The feedback report contained a discussion of the clinical situation and the concept of CD. From the CVi and CVa in the literature, the true CD was given in the feedback report as 14% for BG and 12% for HbA1c.
A parametric t-test and ANOVA were used for comparisons of responses among the countries (because the data generally and for each country showed a gaussian distribution). The Pearson test was used for correlation analysis.
norway
In Norway, the study included GPs participating in 2 different EQAS performed by NOKLUS (one for BG and one for HbA1c). Approximately 70% of the GPs returned the questionnaires. The case history and questionnaire on DM diagnosis (Fig. 1
, patient A) was mailed to all Norwegian GPs participating in the NOKLUS system in October 1998 (almost all GPs in Norway). At this point, the new criteria for diagnosing DM had not been implemented, and the old criterion specifying a fasting capillary glucose of 6.7 mmol/L as the cutoff was used, in contrast to the new criterion specifying 6.1 mmol/L. The Norwegian data are interpreted accordingly. A corresponding survey was performed in November 1997 for HbA1c with quality assessment materials, a case history (Fig. 1
, patient B), and the questionnaire. For the case history, 262 of 444 GPs analyzed the control material in their office laboratory, and the others were given a preset value of 9.4% for HbA1c, with a resulting mean HbA1c value of 9.1%. Details on the materials and methodology for this HbA1c survey have been published previously(4).
the netherlands
GPs were recruited from the region covered by the participating laboratory (St. Elisabeth Hospital, Tilburg). The survey letter was sent by mail to 168 GPs in May 2002; the response rate was 36%.
sweden
Sweden participated only in the HbA1c survey. The HbA1c questionnaires were distributed in August 2001 to 249 primary healthcare centers in connection with an ordinary national EQAS for HbA1c. The response rate was 83%. The EQAS material had a target value for HbA1c [National Glycohemoglobin Standardization Program (NGSP)] of 8.6%, similar to the mean HbA1c result achieved by the primary healthcare centers in the ordinary HbA1c survey. All measurements of HbA1c performed in Sweden were traceable to the Mono S method. Conversion equations for different regional HbA1c standardization procedures and the IFCC reference method have been published recently(14). From these equations, the relationship between NGSP and Mono S HbA1c concentrations can be described by the equation: HbA1c (NGSP) = 0.92 HbA1c (Mono S) + 1.33(14)(15). This relationship has therefore been used to transfer all HbA1c data reported from the Swedish participants in this project to corresponding NGSP concentrations.
hungary
In Hungary, the study questionnaires were posted to 220 GPs randomly selected from the continuous medical education register of GPs all across the country. The response rate was 31%. The study was carried out in collaboration by the Department of Clinical Chemistry of the University of Szeged, the National Institute of Primary Health Care, and the Hungarian Board of General Practitioners. The survey with preset baseline BG and HbA1c values and no quality assessment materials was carried out in the period of February to April 2002. In most laboratories in Hungary, HbA1c values were traceable to the NGSP standard at the time of the survey.
australia
In Australia, the Department of General Practice of the University of Adelaide sent the 2 questionnaires to 300 GPs. The questionnaires were posted to the GPs in June 2002, and 110 were returned, giving a 37% response rate. GP participants were selected randomly through Divisions of General Practice, both urban and rural, within South Australia.
spain
In Spain, the questionnaires were sent in May 2002 to all participating laboratories in the external quality assessment program performed by the Spanish Society of Clinical Chemistry and Molecular Pathology (SEQC). A total of 296 responses were received, of which 291 were included in this study (15.7% of all possible responses). Every laboratory received 2 questionnaires with an explanatory cover letter. Each laboratory was asked to recruit 2 doctors involved in diabetes care to respond to the questionnaire. Responses were returned by participating laboratories to the Secretary of SEQC together with the EQAS monthly response. The period to receive the responses was 4 months, and no reminders were sent. Participating doctors were 61% GPs and 35% endocrinologists. Data from endocrinologists were excluded from the analysis.
In Spain, at the time of the study, clinical guidelines used for doctors followed the old criterion for diagnosing DM, specifying a fasting capillary glucose of 6.7 mmol/L, and the Spanish data must be interpreted accordingly.
south africa
South Africa initially participated, but because of logistic problems, monetary restrictions, and continuous emigration of GPs, only 29 valid responses (response rate, 7%) were received, and the data were excluded from further analysis and comparison.
|
Results |
|---|
|
TopAbstractIntroductionMaterials and MethodsResultsDiscussionReferences |
|---|
. Most GPs were 30–70 years of age, with Spain having the lowest mean age (43.5 years). A total of 1881 GPs responded to the BG questions and 1102 to the HbA1c questions. A total of 55 answers were obviously wrong and were excluded from analysis. Response rates showed considerable variation among countries and were highest in Norway and Sweden.
|
The responses of the GPs rating the importance of repeated BG measurement for questionnaire case patient A are displayed in Fig. 2
. Hungary reported the highest score of 7.6 (mean) and The Netherlands the lowest of 4.3. The overall country results were significantly different (P <0.001); comparisons of means, stratified by country, also showed differences (Fig. 2
).
|
The cumulative reported absolute CDs for BG and HbA1c as percentage deviations from the preset value, stratified by country, are shown in Figs. 3
and 4
. As can be seen from Figs. 3
and 4
, GPs from all countries responded similarly to all questions except when the decrease in HbA1c had to be estimated. We were unable to demonstrate any correlation between the responses on importance of BG retesting and the responses on clinically significant changes in BG values, including all results, for individual countries or in relation to GP age. However, within each country there was wide variation among the responses of individual GPs, pointing to practice variations of possible clinical significance within each country.
|
|
The analytical quality specifications based on CDs were calculated for BG and HbA1c, and the results are summarized in Tables 2
and 3
. Results calculated with 80% and 95% confidence levels are included. As shown in Table 2
, median responses for BG from the different countries were very similar. The HbA1c responses (Table 3
) showed the same pattern as for BG but with more variation among countries. CVa calculations showed lower changes in values and different interpretation patterns for increased HbA1c compared with decreased HbA1c in all countries. Compared with participants in the other countries, participants in Spain and Hungary responded with significantly higher percentage decreases. In Norway and Sweden, participants also performed HbA1c assays to obtain the baseline value, but the responses from these were not different from other countries.
|
|
|
Discussion |
|---|
|
TopAbstractIntroductionMaterials and MethodsResultsDiscussionReferences |
|---|
The main goal in diabetes care is to keep BG and HbA1c as close to reference values as possible. Recommendations for BG concentrations in daily diabetes care are almost identical among countries. The issue of HbA1c is more complex because differences in methods of analysis and standardization may lead to different cutoffs for treatment recommendations(16)(17)(18)(19). However, the survey questions focused on changes in HbA1c and not on absolute values of HbA1c; therefore, country responses can be compared in a relevant manner.
As pointed out in the literature(7)(20), the semantics of the questions is of great importance when interpreting answers to survey questions. We phrased the BG and HbA1c questions so that neither related to a specific confidence level; we therefore used both 80% and the conventional 95% level when performing calculations for both case histories. We think 80% is closer to the confidence level used by GPs in the 2 situations.
The first question concerning patient A raised the issue of retesting or performing additional tests in diagnosing DM. The BG value 5.8 mmol/L is only 0.3 mmol/L below the diagnostic limit for DM in capillary whole blood(2) and is well within possible error limits of point-of-care (POC) instruments used in GP offices(21)(22)(23). We would therefore expect most GPs to recommend additional tests in this situation (preferably in a hospital laboratory), as best shown in the responses from Hungary (Fig. 2
). Some GPs may have "remembered" the diabetes diagnostic limit of 7.0 mmol/L for venous plasma/serum and made their interpretations accordingly, i.e., they did not recommend additional tests. If so, this finding exemplifies a problem of establishing the diagnosis of DM based on criteria using different types of specimens. In a case-finding situation such as this, the use of a capillary specimen is common in all countries in this study. When a DM diagnosis is to be made, a venous sample should be used. The importance of specimen type was also emphasized in our feedback report particularly because it may be potentially harmful to the patient if a diagnosis of DM is postponed.
In the feedback reports we stated that BG measured on an instrument with a reasonable quality will have a CD of 14% and a CVa of 4%–5%(10)(11). In the future, this quality should be attainable by POC instruments. At present, however, many POC instruments have higher CVs(21)(22)(23). As can be seen from our data (Table 2
and Fig. 3A
), 
80% of the GPs (including all countries) responded with changes lower than 14% for a decrease in BG from 5.8 mmol/L. Calculations at the 80% confidence level (Table 2
) for increases in BG led to more attainable CVa values in all countries studied. The National Academy of Clinical Biochemistry (NACB) recommendations(24) suggest that glucose should be measured with an imprecision <3.5% and a bias <2.5%. An imprecision of 3.5% is close to what the most demanding 25% of the GPs expected when using a confidence level of 80% (Table 2
), and an imprecision of 3.5%, including random bias, therefore seems to be in accordance with the analytical expectations of the GPs. Use of a 95% confidence level in calculation of CVa thus leads to values that will be impossible to calculate (Table 2
). For increases in BG (Fig. 3B
), the same percentage of GPs (80%) responded with changes <14%, with Norway and The Netherlands accounting for the lowest percentages. These results demonstrate that most GPs seem to interpret changes in BG when using POC instruments in accordance with quality specifications set by the NACB. Similar expectations of high instrument quality have also been shown in other studies on BG, with doctors and patients as responders(5)(25).
GPs with high (Hungary) or low (The Netherlands) scores on the question of retesting did not seem to set other cutoffs when judging the differences between consecutive BG results. The retesting pattern therefore does not seem to be linked to considerations of analytical and biological variation, but rather to clinical judgment of borderline BG values and awareness that capillary samples in this situation are not appropriate for confirming or ruling out the diagnosis of DM.
For interpretation of HbA1c in monitoring DM, a CD of 12% was quoted as reasonable in the feedback to participants(13), assuming a CVi of 4.0% and a CVa of 3%–4%(26). The NACB guidelines(24) state that HbA1c should have an interassay imprecision <5% or ideally <3%. An imprecision of 3%–5% is in accordance with the quality demands for 50% of the GPs using a confidence of 80% for an increase in HbA1c, except for Hungarian GPs, who expected a lower quality (Table 3
). The quality demands were lower for a decrease in HbA1c. Interpretations at the 95% level were unrealistically strict for up to 50% of the GPs for decreases in HbA1c and for almost all of the GPs for increases in HbA1c (Fig. 4
and Table 3
). Similar results have been shown when patients assess changes in HbA1c(6). Thus, GPs act at a lower threshold when HbA1c increases than when it decreases, probably because increases in HbA1c indicate an increased risk of future diabetes complications.
What lessons can be learned from comparing laboratory data interpretation skills of GPs in different countries? It was recently demonstrated that there is wide variation among laboratory professionals regarding how they interpret test results, in line with our findings of variation among GPs, and that inappropriate interpretation may do harm to patients(27). However, we found that GP responses were surprisingly similar, seemingly irrespective of differences in social, cultural, and organizational aspects of the healthcare systems of the countries involved in this survey. This finding suggests that the method(9) of using familiar situations for case histories for P-EQAS is fairly robust, even when applied in different countries. Similar studies using case histories and responses collected in P-EQAS may be used to establish international clinically acceptable analytical quality specifications. However, some of the response rates and the South African experience underline the need for an organization to deal with P-EQAS.
We believe that our results support the establishment of a P-EQAS of similar design, both for clinicians and laboratory professionals. Such schemes may also serve as an educational tool and a communication channel among laboratories and GPs and could be used to actively disseminate evidence-based best practice. Our study demonstrates the feasibility of performing P-EQAS on an international scale.
Finally, the study highlights the need for better communication with the users of laboratory results and increased awareness by clinicians of how analytical and biological variation can influence interpretation of results. Improved laboratory reports could be instrumental in this function. Research is needed to investigate how better reporting and P-EQAS could reduce the misinterpretation of laboratory test results and thus improve patient and healthcare outcomes.
|
Acknowledgments |
|---|
|
Footnotes |
|---|
|
References |
|---|
|
TopAbstractIntroductionMaterials and MethodsResultsDiscussionReferences |
|---|
The following articles in journals at HighWire Press have cited this article:
|
|
 |
|
  K. M. Aakre, G. Thue, S. Subramaniam-Haavik, T. Bukve, H. Morris, M. Muller, M. V. Lovrencic, I. Plum, K. Kallion, A. Aab, et al. Postanalytical External Quality Assessment of Urine Albumin in Primary Health Care: An International Survey Clin. Chem., October 1, 2008; 54(10): 1630 - 1636. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
A.-H. Kristoffersen, G. Thue, and S. Sandberg Postanalytical External Quality Assessment of Warfarin Monitoring in Primary Healthcare Clin. Chem., October 1, 2006; 52(10): 1871 - 1878. [Abstract] [Full Text] [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
