Outcomes Assessment for Point-of-Care Testing

Department of Laboratory Medicine, Yale University School of Medicine, CB 506, Yale-New Haven Hospital, 20 York Street, New Haven, CT 06504, Fax 203-688-3725, E-mail petrie.rainey{at}yale.edu

"Outcomes Assessment" has become one of the enduring buzzwords of the decade. But it often seems to be like the weather: Everyone talks about it, but relatively few do anything about it. As clinical laboratory scientists, we have repeatedly challenged ourselves to demonstrate that laboratory results affect patient outcome. Yet very few well-done studies have been published. (How many times have you heard and read the results of those few studies by now?)

One difficulty is the universal use of laboratory testing. Would it be acceptable to deny laboratory testing to a control group? The field of point-of-care testing (POCT) offers a major opportunity, because there are many tests not yet in widespread use. This allows ethical comparisons to be designed. (Imagine presenting to the Human Investigation Committee a study of the benefit of digoxin monitoring using a control group in which no digoxin measurements were allowed. Would they approve it? Such a study was done when digoxin assays were just being introduced and convincingly demonstrated a 50% reduction in clinical digitalis toxicity in monitored patients (1).)

Another barrier is the tendency to focus on medical outcomes: decreased mortality or morbidity, shorter length of stay, decreased readmission rate, improved quality of life, etc. Medical interventions may directly alter medical outcomes, but laboratory tests do not. The effect of a test result is always filtered through the change in medical management it engenders. The ability to link the test result to the outcome requires a good correlation between the result and the therapeutic intervention. The size and duration of studies needed to establish the effectiveness of medical interventions are well known. How much larger and longer must a study be to average out the additional variation in the test itself and in the linkage between result and intervention? (Despite such difficulties, the Diabetes Control and Complications Trial clearly demonstrated reductions in long-term complications of diabetes for patients who used self-testing to maintain tight control of blood glucose concentrations (2).)

Most of us are probably not prepared to undertake an outcome study of the size of the Diabetes Control Trial. This should not prevent us from undertaking outcome studies. The trick is to start small. We can focus on direct and short-term outcomes, ones that are more closely linked to the testing event and therefore are less subject to confounding by additional variables. Such studies are manageable in a single institution and can demonstrate outcomes that are meaningful, either because they have intrinsic value or because they can be linked to broader outcomes (for example, one study has shown that POCT can substantially shorten the time to achieve therapeutic heparin concentrations (3), whereas another study has suggested that the risk of recurrent venous thromboembolism increases with the time required to achieve a therapeutic concentration (4)).

Although medical outcomes are the "holy grail", they are not the only outcomes that may be improved. Service outcomes may also result from the introduction of POCT and may be easier to demonstrate than medical outcomes. These may be thought of as improvements in satisfaction, either for the patient or for the caregivers. Financial outcomes are of obvious importance and have the potential to be precisely quantified, a major advantage in comparison studies.

The article by Kilgore et al. (5) in this issue is focused on a very direct outcome of POCT turnaround time. Improved turnaround time is the primary argument for introducing testing at the point of care. Kilgore et al. provide a direct, quantitative comparison of the three primary strategies for reducing turnaround time: central laboratory stat testing, satellite laboratory testing, and POCT. Although they measured traditional analytical turnaround times, they focused on the "therapeutic turnaround time", the time between the decision to test and the initiation of a therapeutic intervention. This is the only meaningful turnaround time for medical outcomes: if there is no change in management, there will be no change in outcome.

Kilgore et al. showed that the therapeutic turnaround time was shortest for POCT, slightly longer for the satellite laboratory, and longest for the central laboratory. Although they suggest that this finding is intuitively obvious, it remains quite important because it was demonstrated empirically and not merely estimated. It was intuitively obvious to many that introducing POCT for electrolytes to the Emergency Department would shorten average length of stay. Estimations of the potential benefits were proffered (6)(7). But when this intuitively obvious hypothesis was tested empirically by Parvin et al. (8), no improvement was observed. (Because central laboratory turnaround times were quite good in this study, the findings may not apply to other settings with a less responsive central laboratory.)

Kilgore et al. also evaluated the service outcome of staff satisfaction and found that the satellite laboratory produced the greatest overall satisfaction, offering timely, accurate answers while minimizing labor for the medical staff. They did not report any financial comparison of the three approaches.

A strong point of these studies was the careful reporting of institution-specific factors that affected the findings. For example, they noted that besides physical distance, another barrier to improving central laboratory therapeutic turnaround time was a cumbersome ordering procedure that was not necessary when testing was done at the point of care or in the satellite laboratory. The failure to identify important institution-specific factors has been a shortcoming in some other studies of POCT. Local needs and resources are often dominant factors in decisions to introduce various forms of POCT. For those who would like to assess the applicability of published studies to their own situation, inclusion of such information is critical.

Some of their secondary findings are fully as interesting as the primary ones. For example, results from the satellite laboratory were most likely to lead to treatment changes, providing a good justification for the greater resources needed to provide this service. Or consider the observation that the largest component of therapeutic turnaround time for POCT was the time required to notify the physician of the result. They found that the scenario of the doctor anxiously waiting for the result of the point-of-care test was uncommon and that most POCT was done in the context of routine monitoring.

It was suggested that POCT may be most efficient when done in the context of a protocol in which specified interventions followed immediately upon receipt of the result. The increasing use of such management protocols and clinical pathways should facilitate outcome studies for laboratory testing, because they ensure the correlation between result and intervention. The importance of such protocols is suggested by the widely cited studies of Despotis and co-workers (9)(10), demonstrating decreased operative time, decreased blood product usage, and decreased chest tube drainage with the use of bedside coagulation testing during cardiopulmonary bypass. The use of a management algorithm was felt to be a critical component in achieving the benefits.

The current study of Kilgore et al. (5), as well as the classic studies of Parvin et al. (8) and Despotis et al. (9), illustrate a number of principles that can help in the design of a manageable and meaningful outcome study:

(a) Do a real observational study, not a decision analysis study (the latter is an extended thought experiment, with conclusions that are totally dependent on the initial assumptions and that may be invalid if a contributory variable is overlooked in designing the model).

(b) Do the study prospectively. If possible, use a randomized or crossover design.

(c) Minimize other sources of variability: limit the study to a single service, diagnosis, or procedure.

(d) To increase sensitivity for effects on medical outcomes, use management algorithms that assure consistent interventions in response to test results.

(e) Choose outcome measures that may be quantitated (e.g., length of stay or units of red cells transfused).

(f) Use outcome variables whose values become determined in close temporal proximity to the testing event (e.g., turnaround time or Emergency Room length of stay), thereby minimizing opportunity for contamination by confounding events.

(g) Use outcomes that may be surrogate markers for longer-term outcomes (e.g., chest tube drainage).

(h) Choose outcomes that have intrinsic financial value (e.g., decreased use of blood products or decreased Operating Room time).

(i) Identify institution-specific factors that may have affected the results (e.g., turnaround time for central laboratory testing).

The field of POCT is young enough that there are few tried-and-true ways of doing things. We are all experimenting. By doing our experiments in an organized, hypothesis-driven manner and sharing our findings through publication, we can begin to establish what works and what is beneficial.

The opportunity to answer some of these outcome questions may be short-lived. Bedside glucose testing is already in widespread use. Given the large amounts of money and effort being expended for this, it would be comforting to have convincing evidence that bedside testing of hospitalized inpatients provides better glycemic control than testing in the central laboratory, or that it results in faster stabilization of insulin dosing and faster discharges. The problem today will be to find an appropriate control group.

For most other recently introduced point-of-care tests, there are still plenty of opportunities. Clinical laboratory scientists should be taking advantage of these opportunities, and POCT manufacturers should be supporting those efforts. Medicine is becoming ever more evidence-based and outcome-driven. Such information is needed to optimize our use of decreasing medical resources. Moreover, there is always the concern that lack of evidence of a benefit (i.e., no one has done the study) will be taken as evidence of lack of a benefit.

References

1. Koch-Weser J, Duhme DW, Greenblatt DJ. Influence of serum digoxin concentration measurements on frequency of digitoxicity. Clin Pharmacol Ther 1974;16:284-287. [Web of Science][Medline] [Order article via Infotrieve]
2. . The Diabetes Control and Complications Trial Research Group. The effect of intensive treatment of diabetes on the development and progression of long-term complications of insulin-dependent diabetes mellitus. N Engl J Med 1993;329:977-986. [Abstract/Free Full Text]
3. Becker RC, Cyr J, Corrao JM, Ball SP. Bedside coagulation monitoring in heparin-treated patients with active thromboembolic disease: a coronary care unit experience. Am Heart J 1994;128:719-723. [Web of Science][Medline] [Order article via Infotrieve]
4. Hull RD, Raskob GE, Brant RF, Pineo GF, Valentine KA. Relation between the time to achieve the lower limit of the APTT therapeutic range and recurrent venous thromboembolism during heparin treatment for deep vein thrombosis. Arch Intern Med 1997;157:2562-2568. [Abstract/Free Full Text]
5. Kilgore ML, Steindel SJ, Smith JA. Evaluating stat testing options in an academic health center: turnaround time and staff satisfaction. Clin Chem 1998;44:1597-1603. [Abstract/Free Full Text]
6. Sands VM, Auerbach PS, Birnbaum J, Green M. Evaluation of a portable clinical blood analyzer in the emergency department. Acad Emerg Med 1995;2:172-178. [Web of Science][Medline] [Order article via Infotrieve]
7. Tsai WW, Nash DB, Seamonds B, Weir GJ. Point-of-care versus central laboratory testing: an economic analysis in an academic medical center. Clin Ther 1994;16:898-910. [Web of Science][Medline] [Order article via Infotrieve]
8. Parvin CA, Lo SF, Deuser SM, Weaver LG, Lewis LM, Scott MG. Impact of point-of-care testing on patients' length of stay in a large emergency department. Clin Chem 1996;42:711-717. [Abstract/Free Full Text]
9. Despotis GJ, Santoro SA, Spitznagel E, Kater KM, Cox JL, Barnes P, Lappas DG. Prospective evaluation and clinical utility of on-site monitoring of coagulation in patients undergoing cardiac operation. J Thorac Cardiovasc Surg 1994;107:271-279. [Abstract/Free Full Text]
10. Despotis GJ, Joist JH, Goodnough LT. Monitoring of hemostasis in cardiac surgical patients: impact of point-of-care testing on blood loss and transfusion outcomes. Clin Chem 1997;43:1684-1696. [Abstract/Free Full Text]