| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
|
|
|
|||||||
|
|||||||||
Evidence-Based Laboratory Medicine and Test Utilization |
1 Institute for Medical Technology Assessment (iMTA), Erasmus MC, Rotterdam, The Netherlands.
2 Department of Pediatric Surgery, Sophia Children’s Hospital, Erasmus MC, Rotterdam, The Netherlands.
3 Institute of Medical Informatics (MIEUR), Erasmus MC, Rotterdam, The Netherlands.
4 Netherlands Institute for Health Services Research (NIVEL), Utrecht, The Netherlands.
aAddress correspondence to this author at: institute for Medical Technology Assessment (iMTA), Erasmus MC, P.O. Box 1738, 3000 DR Rotterdam, The Netherlands. Fax 31-10-408-9092; e-mail m.poleij{at}erasmusmc.nl.
 |
Abstract |
|---|
 Top Abstract IntroductionMaterials and MethodsResultsDiscussionReferences |
|---|
Methods: We installed the CDSS in 118 practices [159 general practitioners (GPs)] throughout The Netherlands and calculated the costs of the intervention in this group. During a period of 6 months before and 6 months after installation of the CDSS, the test-ordering behavior of 87 (109 GPs) of these 118 study practices was studied and the results were compared with those of a nonhistorical control group that did not receive the CDSS. In addition the costs of laboratory requests were calculated for both groups.
Results: Total intervention costs, comprising development costs and installation costs, amounted to 
79 000 (670 per practice). Whereas the introduction of the CDSS did not affect the number of order forms submitted to the laboratories, it did reduce the number of blood tests per order form. As a result, the CDSS yielded mean savings on the costs of laboratory requests of 847 per practice per 6 months.
Conclusions: This study demonstrates that providing electronic decision support for ordering blood tests in primary care represents an economically promising concept. Savings on laboratory costs are achievable and not offset by disproportionally high intervention costs.
|
Introduction |
|---|
|
TopAbstractIntroductionMaterials and MethodsResultsDiscussionReferences |
|---|
Several questions regarding the efficiency of initiatives to improve test ordering remain unanswered, however. For example, such interventions can be very costly to develop and implement. A study on physician education with feedback, performed in a hospital setting, indicated that the cost of the interventions might have canceled out any potential savings on hospital costs (19). Therefore savings on laboratory tests must be balanced against such program costs, which only few studies have calculated (9)(20).
Clearly, cost-effectiveness of interventions aimed at guiding test-ordering behavior is not yet fully understood. Hence, this study evaluated the cost consequences of a computer-based decision-support system (CDSS) for ordering blood tests in a primary care setting. This CDSS was based on an almost identical system that had been tested in a small regional study in The Netherlands, with promising results (17)(18)(21).
|
Materials and Methods |
|---|
|
TopAbstractIntroductionMaterials and MethodsResultsDiscussionReferences |
|---|
intervention
The intervention consisted of a CDSS used for ordering laboratory blood tests and integrated into the computer-based patient record. The GP must first select the indication, from a list of indications grouped by clinical guidelines, that most closely fits the patient’s complaints. The CDSS then shows an optimal but restricted list of blood tests based on the recommendations for blood test ordering from the guidelines of the Dutch College of General Practitioners (http://nhg.artsennet.nl/). The GP could adhere to the proposed list or add or remove tests from the list. Finally, the CDSS updated the patient record and printed a patient-specific order form for the patient to deliver at the laboratory. Screenshots of the CDSS are available online [see Figs. 1 and 2 in the Data Supplement that accompanies the online version of this article at http://www.clinchem.org/content/vol53/issue2].
study design and data collection
The study started with a 6-month preintervention observation period. During this period, the GPs ordered laboratory tests with their conventional method, which is to tick off the blood test(s) on a paper order form that contains a large list of possible tests. The only difference with common practice during the preintervention period was that to enable further analysis the GPs recorded the indication for the test on the conventional form. Then, during the 6-month intervention period, which began immediately after the CDSS had been installed, the GPs were asked, but not obligated, to use the CDSS. Paper order forms remained available during the entire intervention period.
This study aimed to compare the economic consequences of physician test-ordering behavior in the preintervention and the intervention periods. Data were analyzed according to the intention-to-treat principle, i.e., data for all physicians were analyzed, regardless of whether or not they had used the CDSS and regardless of the amount of use. To control for the possibility of autonomous changes independent of the introduction of the CDSS, we created a control group comprising a nationally representative sample of primary care practices (n = 47). Test-ordering data from this group were also collected for both the preintervention and the intervention period. In summary, the study design can be characterized as a pretest-posttest design with a control group (Fig. 1
).
|
outcome measures
To explore the economic consequences of introducing the newly developed CDSS, we calculated both the intervention costs and the costs of laboratory requests.
intervention costs
We estimated minimum, maximum, and base-case intervention costs, comprising the costs of both developing and installing the CDSS. Regarding the development costs, we included the personnel costs for reviewing 83 different guidelines for possible recommendations on blood tests, writing the content of the CDSS, having a panel of experts judge the content, programming the software, testing prototypes, writing an explanatory leaflet about the CDSS, and writing instructions for installation and use. Calculations of personnel costs were based on income scale 10 of the Dutch Collective Employment Agreement for University Hospitals. Taking into account public holidays, vacation, illness, and study leave, the number of working hours per person per year was set at 1540. This calculation method resulted in hourly personnel costs of 30, including an increment of 35% for holiday allowances and social security expenses.
Installation costs comprised personnel costs and travel costs. Following a bottom-up approach, we arrived at a minimum estimate of the installation costs. In addition to the above-mentioned personnel costs of 30 per h for members of our team, this minimum estimate included costs for the installation activities performed by GPs. Based on the Dutch Collective Employment Agreement for Health Centers, these costs were set at 40 per h. Moreover, costs of transportation by car (0.12 per kilometer) were included. This bottom-up estimate, however, did not allow for unplanned additional work, such as extra coordination, failed installations, and GPs canceling appointments for installation. Therefore, a top-down strategy was pursued to make a maximum estimate. This approach involved studying the total number of hours per week that several members of our team spent installing the CDSS throughout the study period.
costs of laboratory requests
The costs of laboratory requests depended on both the number of blood samples collected and the number and type of laboratory tests performed. Data on the number of blood samples and blood tests were obtained from the laboratories, and costs were calculated by multiplying the number of blood samples and the number of tests by their respective unit costs. In line with Dutch guidelines on economic evaluations of healthcare (22), unit costs were obtained from the national list of charges established by the Dutch Board for Health Care Tariffs for the year 2003. The cost for obtaining a blood sample was 11.50. The cost per test varied between 1.47 and 33.19, depending on the type of test. In addition to these costs, which included the costs of material, laboratory personnel, and housing, we added the salary costs of the clinical chemist or medical microbiologist who headed the laboratory but was not employed by the laboratory.
statistical analyses
We used the 
2 test to compare baseline characteristics of the participating physicians from the intervention group and from the control group and the t-test for 2 independent samples to analyze differences between both groups in the costs of laboratory requests. Two-sided P values <0.05 were considered statistically significant.
|
Results |
|---|
|
TopAbstractIntroductionMaterials and MethodsResultsDiscussionReferences |
|---|
).
|
intervention costs
The minimum and maximum estimate of the costs of developing the CDSS were 41 000 and 48 000, respectively, and the base-case estimate was 44 000 (Table 2
). These costs included writing the content of the CDSS (for which 14 to 21 working days were required), holding 3 expert meetings (counting 1.5 days per meeting for each attending researcher and for each expert), programming the software (60 to 80 days), testing prototypes (8 days), and writing instructions (50 days).
|
Our team carried out the installation in 90 practices. In another 8 practices, the physicians themselves accomplished the installation. A few of these practitioners also installed the CDSS in practices (n = 20) of other participating physicians. According to the bottom-up approach, the minimum estimate was a mean of 153 installation costs per practice. The top-down calculation revealed that our team spent 130 working days on performing installations and providing assistance. Using this figure, we calculated the mean total costs of installation at 434 per practice. The base-case estimate was the average of these 2 estimates (293 per practice).
Total intervention costs were calculated to be 59 000 to 99 000 (502 to 839 per practice), with a base-case estimate of 79 000, or 670 per practice (Table 2
).
costs of laboratory requests
Analysis of the costs of laboratory requests for both the intervention and the control group during the preintervention and the intervention periods (Table 3
) revealed no evidence that the introduction of the CDSS had a statistically significant effect on the number of order forms (+0% in the intervention group vs +2% in the control group; P = 0.44). The number of tests per order form decreased in the intervention group, however, whereas it remained practically unchanged in the control group (–6% vs +0%; P = 0.001). Overall, we found a mean cost decrease of 3% (639) in the intervention group, compared with a 2% (208) increase in the control group (P = 0.09). This result suggests that the CDSS yielded mean cost savings of 847 per practice per 6 months (i.e., –639–208).
|
break-even point
The break-even point, at which the savings on laboratory costs exceeded the intervention costs, was reached after 5 months. We performed a sensitivity analysis, repeating these calculations with combinations of (a) the 95% confidence interval of the difference in the cost of laboratory requests between the intervention and the control group and (b) the minimum and maximum estimate of the intervention costs. In the best-case scenario—using the upper limit of the confidence interval of the difference in the cost of laboratory requests (yearly savings of 3 669 per practice) and the minimum estimate of the intervention costs (502 per practice)—the intervention costs would be offset by savings as early as 2 months after beginning the intervention. In the worst-case scenario, the cost of laboratory requests would actually increase by 282 per year in the intervention group compared with the control group, so that the investment (at its maximum of 839 per practice) would not be outweighed by savings on laboratory costs at all.
|
Discussion |
|---|
|
TopAbstractIntroductionMaterials and MethodsResultsDiscussionReferences |
|---|
) or the number of order forms and number of tests per order form (data not shown). The GPs were asked to fill in the indication for the test on the order form during the preintervention period and thus were already aware of being studied. This awareness did not seem to make them change their test-ordering behavior, according to analysis of their laboratory use data (and that of the control group) in the preintervention period compared to the 6 months preceding the preintervention period (data not shown).
Our investigation of the costs of the CDSS was limited in certain aspects. First, costs of future maintenance were not included. Note that we implemented the CDSS in a relatively small group of practices dispersed all over the country. Broader implementation would enable considerable economy of scale in initiating and maintaining the system. Second, we did not actually measure the time required to order laboratory tests and the time used for consulting guidelines on laboratory testing with the new CDSS, which may differ from those required with the traditional test-ordering method. To get impressions of the potential additional costs caused by using the CDSS and of the level of satisfaction with it, the GPs were asked to complete a questionnaire (see Table 1
in the online Data Supplement). Introducing the CDSS required some additional effort for the GPs, a finding consistent with previous research in different contexts that suggested that it may take physicians more time to write orders with the computer than with a paper form (23)(24)(25). Furthermore, it may be troublesome for laboratories to process this new way of ordering tests. The laboratories in this study generally were rather unsatisfied with physician submission of new computer-generated order forms, as data collected by a questionnaire revealed (see Table 2
in the online Data Supplement). This finding suggests that before the CDSS can achieve its full potential some practical difficulties encountered by the laboratories need to be addressed, a process that may involve a range of solutions varying from relatively low-cost methods (e.g., ensuring that new order forms can be read optically in the usual way) to costlier ones (e.g., enabling electronic sending and receiving of requests for laboratory tests). Costs for these procedures may drop in the long run, however, and make the process of test ordering less sensitive to error.
Use of the CDSS may also be affected by financial aspects and incentives. In The Netherlands, laboratory testing in primary care is included in standard care covered by the mandatory health insurance. Therefore, any cost savings generated by the CDSS accrue to society as a whole, in the form of a decrease (though small) in healthcare costs. Dutch GPs have no financial interest in laboratory testing, because they receive no separate reimbursement for laboratory tests ordered. On the contrary, implementing the CDSS may have negative consequences for the laboratory’s financial stability because the CDSS appeared not to lower the number of blood samples but did lower the number of tests per order form, and because the number of blood samples analyzed is a more important cost-driving factor than the number of tests requested. Because laboratories are partly reimbursed per blood sample and partly per test, their incomes may decline at a faster rate than their costs.
Reduction in test ordering may be associated with substitution of care, for example a shift to other, perhaps more expensive healthcare procedures to reduce diagnostic uncertainty or to reassure patients (26)(27). With this in mind, we performed a preliminary analysis into possible substitution effects. We found no clear evidence that a decrease in laboratory requests was accompanied by a shift to more prescribed medications, GP consultations, or referrals to specialist care (see Table 3
in the online Data Supplement). Because of the small numbers involved, no firm conclusions are as yet possible on this issue. This finding nevertheless seems to confirm the few earlier studies on substitution effects reported in the literature. None of these studies found that a decrease in test requests went together with an increase in other forms of medical care (14)(28).
The ultimate goal of implementing a decision-support system is to improve quality of care and patient health. Undeniably, however, strategies targeted at optimizing laboratory test use may have unclear or negative consequences, because such interventions could lead to the ordering of fewer tests than are necessary. We were unable to study such effects within the framework of this investigation, but the CDSS was implemented with the explicit aim of making the GPs more aware of the recommendations for test ordering from the guidelines of the Dutch College of General Practitioners, which are evidence-based and considered authoritative. Importantly, compared with the preintervention period, the participants departed less often from these guidelines during the intervention period (data not shown). These findings strongly suggest that our intervention enhances the quality of care, leading to the further conclusion that the intervention is likely to eventually produce positive effects on patient health.
In summary, providing electronic decision support for ordering blood tests in primary care represents an economically promising concept. Our study indicates that implementing the CDSS generates savings on laboratory costs, which are not offset by disproportionally high intervention costs.
|
Acknowledgments |
|---|
|
References |
|---|
|
TopAbstractIntroductionMaterials and MethodsResultsDiscussionReferences |
|---|
The following articles in journals at HighWire Press have cited this article:
|
|
 |
|
  X. Bossuyt, K. Verweire, and N. Blanckaert Laboratory Medicine: Challenges and Opportunities Clin. Chem., October 1, 2007; 53(10): 1730 - 1733. [Abstract] [Full Text] [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
