| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
|
|
|
|||||||
|
|||||||||
Evidence-Based Laboratory Medicine and Test Utilization |
1 NOKLUS, Norwegian Quality Improvement of Primary Care Laboratories, Laboratory of Clinical Biochemistry, Haukeland University Hospital, Bergen, Norway.
2 NOKLUS, Section for General Practice, Department of Public Health and Primary Health Care, University of Bergen, Bergen, Norway.
aAddress correspondence to this author at: NOKLUS, Norwegian Quality Improvement of Primary Care Laboratories, University of Bergen, Boks 6165, 5892 Bergen, Norway. Fax 47-5558-6710; e-mail gunn.kristensen{at}isf.uib.no.
 |
Abstract |
|---|
 Top Abstract IntroductionMaterials and MethodsResultsDiscussionReferences |
|---|
Methods: From October 2001 through March 2004, we conducted 6 glucose surveys for diabetes patients and coordinated them with an EQAS for office laboratories. Patients received 2 control samples by post twice a year. They measured each control sample in duplicate in accordance with written instructions, returned the results, and received an assessment of their analytical performance. Participants who got a poor evaluation were followed up by phone and were offered guidance.
Results: Participating in an EQA program over a period of 3 years decreased the percentage of poor results among diabetes patients significantly, from 11.2% to 1.9% in the first and last surveys, respectively. Between-participant CVs improved from 5.5% to 3.7% and were comparable to results from office laboratories. It was difficult to sort out factors contributing to quality improvement.
Conclusions: Implementing a traditional EQAS among diabetes patients may improve the analytical quality of SMBG and could be convenient for motivated patients. Further evaluation of the clinical usefulness of implementing such a program is needed, however, and costs as well as limitations of current EQAS for glucose in general should be taken into account.
|
Introduction |
|---|
|
TopAbstractIntroductionMaterials and MethodsResultsDiscussionReferences |
|---|
Patients performing SMBG may theoretically control their instruments in several ways: by comparing results from self-monitoring with results obtained in a hospital laboratory, a doctor’s office, or a pharmacy; by using control materials recommended by the manufacturer; or by participating in an external quality assessment (EQA) program. There is little information about control routines for patients performing SMBG. However, Skeie et al. (9) reported that only approximately one third of patients performed instrument controls of some kind. These controls were performed mainly by duplicate measurements, comparing results from self-monitoring with results obtained at the doctor’s office, or use of several SMBG instruments. Only 6% of the patients performed quality control with commercial control materials.
External quality assessment schemes (EQAS) are important for improving analytical quality (11) and are integrated into the workflow of the Norwegian Quality Improvement of Primary Care Laboratories (NOKLUS). The objective of this study was to investigate whether an EQA program designed for office laboratories could also improve the quality of SMBG measurements among diabetes patients.
|
Materials and Methods |
|---|
|
TopAbstractIntroductionMaterials and MethodsResultsDiscussionReferences |
|---|
1900 laboratories associated with Norwegian general-practice offices. We sent control samples identical to those used in NOKLUS EQAS to the diabetes patients participating in the surveys.
participants
Diabetes patients performing SMBG from 4 counties in Norway were recruited by an advertisement in the Journal of the Norwegian Diabetes Association, by distribution of an enrollment folder at pharmacies, and by an internet link on the Norwegian Diabetes Association home page. The study was designed in accordance with guidelines from the Regional Committee for Medical Research Ethics, and the participants gave informed consent to take part in the study.
Diabetes patients who participated in the first and second surveys (n = 171) were included in the study. Ten patients quit the program during the study period. Approximately 10% of the patients in each survey group used instruments classified to an instrument group with <12 participants (see the section on glucose instruments below), and we did not send control samples to these patients. Thus, 120 to 152 patients in each survey group received control samples, with 34 patients receiving control samples in all 6 surveys. The response rate was 71%–86%.
The characteristics of the participants, including background variables and use of SMBG, are shown in Table 1
. This information was obtained with a questionnaire attached to the last survey (survey 6 in March 2004). Of the 161 patients included in this survey, 126 (78%) responded to the questionnaire. Almost one half of the diabetes patients who were included in the program were self-educated, more than one half performed SMBG more than 10 times per week, and 50% had more than one instrument (we did not ask patients who had more than one instrument to specify whether these were identical or different types).
|
study design
Diabetes patients participating in the EQAS received 2 control samples by post twice a year. Participants measured each control sample in duplicate in accordance with written instructions and returned the results to NOKLUS together with information about the instrument and the lot number and expiration date of the strips used. In response, participants received an assessment of the analytical performance within 1 month. Participants who got a "poor" evaluation were given written instructions on how to use their specific instrument. They were also contacted by phone (by G.B.B.K. or K.N.) and offered guidance and advice concerning the poor result. We focused on a thorough review on measurement performance, including sample volume, calibration code, and storage and expiration date of strips, as well as mixing and applying the control samples properly before use.
control materials
NOKLUS uses 2 different control materials in 2 concentrations (6 and 20 mmol/L) in EQAS for glucose meters, depending on the type of instrument (12)(13): Sugar Chex (Streck Laboratories) and EDTA blood, stabilized with NaI (Sero AS).
The control materials used in our EQAS differ in composition from native capillary blood, and method-specific target values were therefore used. The method-specific target value was calculated as the median of all results from participants who used a particular type of instrument, after outliers had been excluded (14). Stability tests were performed for 5 days in each survey. Mean values did not differ significantly for 3 days, and all samples were analyzed by patients within this period.
glucose instruments
Because we used method-specific target values, only diabetes patients with instruments that could be classified to an instrument group with more than 12 participants received control samples and participated in the EQAS. Thus the following instrument brands were included; OneTouch Basic/II/Profile and GlucoTouch (LifeScan); Accu-Chek Sensor (Roche Diagnostics); Glucometer Dex/Dex2 and Ascensia Elite (Bayer Diagnostics); Precision QID, MediSense Pen, MediSense Card, and Precision Xtra (Abbott Laboratories/MediSense).
The distribution of instruments among the 120 diabetes patients who received control samples in the first survey in October 2001 and the corresponding distribution of these instruments in laboratories affiliated with Norwegian general practice offices at the same time are shown in Table 2
.
|
participant assessment
In our EQAS for the patients, we used the same quality specifications that NOKLUS has used in EQAS for glucose in office laboratories, taking biological variation and clinical needs into account. Participant performance was assessed as "good" if the result (mean of duplicate measurements) deviated from the target interval (the method-specific target value ± 0.1 mmol/L) by <5%; "acceptable" if their result deviated from the target interval by 5% to 10%, and "poor" if the result deviated more than 10% from the target interval. Thus, if the target value is 7.00 mmol/L, the participant will get the assessment "good" if the result is 6.55–7.45 mmol/L, "acceptable" if the result is 6.20–6.54 mmol/L or 7.46–7.80 mmol/L, and "poor" if the result is <6.20 mmol/L or >7.80 mmol/L.
Participant precision was not assessed, but if the difference between 2 measurements of the same control sample was larger than the deviation between the method-specific target value and the acceptable lower limit for the assessment, their results in the EQAS were described as "uncertain".
To compare analytical quality throughout the surveys, we divided participants into 3 groups, and office laboratories constituted an additional group:
Study group 1.
Results from all patients, i.e., patients who were assessed in at least 1 of the 6 surveys (n = 158–218 results in each survey).
Study group 2.
Results from the subgroup of patients from study group 1 who were assessed in all 6 surveys (n = 68 results in each survey).
Study group 3.
Results from the subgroup of patients from study group 1 who were assessed in at least 1 of the 6 surveys and who used instruments that were also used by office laboratories (n = 135–181 results in each survey).
Study group 4.
Results from office laboratories using the same instruments as diabetes patients who were assessed (n = 955–1290 results in each survey).
The percentage distribution of instruments differed between diabetes patients and office laboratories, as exemplified by the results of the first survey (Table 2
). Therefore, to enable an instrument-independent comparison of performance between diabetes patients and office laboratories, we weighted results from the office laboratories according to the percentage distribution of instruments among diabetes patients in each survey. Only instruments used by both patients and office laboratories were included in this procedure; thus, results from patients using the Glucometer Dex were excluded in study group 3 (Table 2
).
statistics
We calculated imprecision, SDwithin, and CVwithin by use of duplicate measurements performed with the same control sample by all participants with a specific type of instrument, based on the formula:
 |
 |
To assess changes in analytical quality, we used the binominal test for proportions, the F-test, and linear regression and compared variances. For each survey, we calculated an estimate of between-participant CV by pooling the CVbetween values for both control samples for all instruments. The between-participant CV and the percentage of poor performances for each survey were used to demonstrate analytical quality. The percentage outliers ranged from 0% to 5%. The level of statistical significance was set to 5%.
|
Results |
|---|
|
TopAbstractIntroductionMaterials and MethodsResultsDiscussionReferences |
|---|
). The between-participant CVs (see Materials and Methods) also decreased throughout the study period, although the decrease was significant only for study group 2 (Fig. 1B
). However, both study groups demonstrated a significantly lower percentage of poor performers and a significantly lower between-participant CV in the last survey compared with the first survey (1.9% compared with 11% and 3.7% compared with 5.5% for study group 1, respectively; and 0% compared with 16% and 3.0% compared with 6.0% for study group 2, respectively).
|
The participants demonstrated good precision in all 6 surveys, with pooled CVwithin values of 2.4% to 3.2%.
comparison of analytical quality between diabetes patients and office laboratories
In the first and second surveys, which evaluated the results obtained with the same kind of instruments, the percentage of poor results was significantly higher for diabetes patients than for office laboratories [14% compared with 8.8% (P = 0.04) and 22% compared with 13% (P = 0.003) for the first and second survey, respectively; Fig. 2A
]. However, for diabetes patients, the percentage of poor assessments decreased throughout the 6 surveys, as demonstrated by linear regression. This decrease was not seen for office laboratories. The same decreasing trend was observed for pooled between-participant CVs (Fig. 2B
). However, from having a significantly higher between-participant CV in the 2 first surveys, i.e., 5.9% compared with 5.4% and 7.7% compared with 6.4% in surveys 1 and 2, respectively, the diabetes patients demonstrated significantly lower between-participant CVs than did office laboratories in surveys 3, 5, and 6 (6.1% compared with 7.2%, 4.8% compared with 6.6%, and 3.9% compared with 4.6%; Fig. 2B
).
|
reasons for poor performance
For 20% of diabetes patients with poor assessments, explanations such as the use of expired test strips or a deviant lot of test strips, not using the right calibration code on the instrument, or not mixing the control samples properly were detected by follow-up of participants. Variability among lots might also contribute to poor analytical quality (12); however, because the number of participants using each lot number was rather small (1–2 participants per lot number), it was impossible to estimate the extent of between-lot variability among the patients. For office laboratories, on the other hand, some lot numbers were detected that gave glucose values that differed significantly from the method-specific target interval. Some of these deviant lot numbers were also used by patients and might be a reason for poor performance.
Among patients who got a poor assessment, those who changed instruments obtained as high a percentage of poor results in the following survey as those who did not change instruments. Similar findings were seen with participants who received good/acceptable assessments in the first survey and then changed instruments. We did not give any recommendations concerning the selection of a new instrument. Approximately 10% of the participants changed instruments between the different surveys.
evaluation of the eqas
To evaluate the EQAS, we attached a questionnaire to the last survey (March 2004). According to the survey results, 81% found it useful to participate in the program, only 5% did not, and 84% indicated that participation in the EQA surveys increased their confidence in SMBG results to some or a high degree. Of the participants who needed follow-up, 90% were very satisfied or satisfied with the follow-up.
Among the survey responders, 90% expressed interest in participating in a similar EQAS on a regular basis, and 65%, 17%, and 16% of these wanted to receive control samples 2, 4, or 1 time a year, respectively. One half were willing to pay for each survey, but not more than US $15.00 (80%) or US $30.00 (15%). The cost of participating in one survey was estimated to be US $60.00, of which approximately US $30.00 is allocated to distributing the control material.
|
Discussion |
|---|
|
TopAbstractIntroductionMaterials and MethodsResultsDiscussionReferences |
|---|
Survey 2 revealed a deterioration of the analytical quality. This finding can be explained by the fact that before this survey Roche Diagnostics introduced an upgraded test strip for the Accu-Chek Sensor. Measurements performed with this test strip gave significantly different values than did measurements performed with the previous test strip. If separate target values were used to assess results for the 2 different strips (12), the percentage of poor assessments and the between-participant CVs would be reduced for both diabetes patients and office laboratories, e.g., the percentage of poor results in Fig. 2A
would be reduced from 21.8% to 11.2% for diabetes patients and from 13.1% to 8.7% for office laboratories. Linear regression analysis of these results indicated an even more significant decrease (P = 0.002) in poor results than that shown in Fig. 2A
for diabetes patients throughout the study. For office laboratories, the results from linear regression did not change.
The within-participant imprecision values obtained in this study were within the quality specifications of <5% set by the ADA and by the patients themselves (7)(15). However, the within-participant CVs should be interpreted with caution because some of the participants might have analyzed the sample more than twice and reported the 2 best results (16). Regarding total error, because we used method-specific target values, it is difficult to compare the results from this study with the quality goals set by the ADA or ISO 15197 (6)(7)(8). The diabetes patients, however, demonstrated improved analytical quality during the study, and at the end of the study the quality was similar to or better than the quality of the office laboratories (Fig. 2
). The results did not indicate that changing to newer instruments improved the analytical quality. Therefore, the manufacturers’ redesign of SMBG instruments to improve their accuracy and precision throughout the study period did not seem to explain the improvement.
Because the participants in this study were selfselected and likely to be particularly capable and motivated, there might have been an overestimation of the quality of the SMGB measurements, especially at the start of the study. In addition, the "Hawthorne effect" (17)—participation in a program highlighting the importance of good analytical quality may be sufficient to cause people to change their behavior—might have played a role. Although it is difficult to sort out these different effects, we still think that an EQAS with a systematic feedback program will improve the quality of the measurements, and the Hawthorne effect would probably decrease over time. Most office laboratories have participated in the EQAS since 1993 with a similar decrease in the percentage of poor results from 17% to 6% throughout the first 6 years (18). The percentage of poor results for office instruments used in this study (with the notable exception of HemoCue users) seems to have plateaued at 10%, which may indicate that the analytical quality achieved is representative of current meter technology in an outpatient setting. Our results are in line with the findings of Rumley (19), who reported that after implementation of an EQAS for glucose in 1993 and improved meter technology, the frequency of unacceptable results fell to 5%–7% in 1997.
A limitation to the EQA program for diabetes patients, as in almost any EQA program, is that artificial control materials may not be commutable to native blood samples. Therefore, aberrant results for the artificial control materials can be difficult to interpret because the results might not reflect the situation when native blood is used and vice versa (12). Another limitation in EQA surveys is that there is no control of blood sampling and no immediate feedback. An alternative approach uses a split-sample design in which glucose measurements performed by patients are compared with a result from a standardized comparative method. This design implies direct observation of user performance; thus, user errors could be more easily detected. Some studies recommend this method for routine use (20)(21)(22)(23). NOKLUS is currently evaluating a model in which patients use the split-sample design to control their SMBG performance at pharmacies or in doctors’ offices. This decentralized model would require more resources than does a traditional EQA program.
In Norway, health authorities have stipulated that all SMBG instruments with test strips should be examined by a standardized procedure that implies testing simultaneously by an experienced technologist and a group of patients before test strips are reimbursed by the government (16). An additional effect of running an EQAS for glucose for diabetes patients is that it will give an opportunity to continuously monitor the quality of SMBG technology on the market. The cost of approximately US $120.00 per year for 2 control samples should be compared with the costs of SMBG, e.g., using on average 4 strips a day, which amounts to approximately US $1600.00 per year. However, the clinical usefulness of implementing such a program should be evaluated further, and costs as well as limitations of current EQAS for glucose in general should be taken into account.
|
Acknowledgments |
|---|
|
Footnotes |
|---|
|
References |
|---|
|
TopAbstractIntroductionMaterials and MethodsResultsDiscussionReferences |
|---|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
) includes results from diabetes patients who received assessment in at least 1 of the 6 surveys (n = 158–218 results), whereas Study group 2 (
) includes results from diabetes patients who received assessment in all 6 surveys (n = 68 results).
; n = 135–181 results) and office laboratories (Study group 4;
; n = 955–1290 results) with the same brands of instruments (see Materials and Methods).