HOME HELP FEEDBACK SUBSCRIPTIONS ARCHIVE SEARCH TABLE OF CONTENTS		QUICK SEARCH:   [advanced] Author: Keyword(s): Year:  Vol:  Page:

This Article Abstract Full Text (PDF) Data Supplement Submit an electronic Letter to the Editor about this paper View responses Alert me when this article is cited Alert me when eLetters are posted Alert me if a correction is posted Citation Map Services Email this article to a friend Similar articles in this journal Similar articles in ISI Web of Science Similar articles in PubMed Alert me to new issues of the journal Download to citation manager Citing Articles Citing Articles via HighWire Citing Articles via ISI Web of Science (17) Citing Articles via Google Scholar Google Scholar Articles by Lim, E. M. Articles by Vasikaran, S. D. Search for Related Content PubMed PubMed Citation Articles by Lim, E. M. Articles by Vasikaran, S. D. Related Collections Laboratory Management

Quality Assessment of Interpretative Commenting in Clinical Chemistry

¹ Clinical Biochemistry, Pathcentre, Perth WA6909, Australia. ² Melbourne Pathology, Collingwood VIC3066, Australia. ³ RCPA-AACB Chemical Pathology QAP, RCPA Quality Assurance Programs Pty Ltd., Flinders Medical Centre, Adelaide SA5042, Australia. ⁴ Chemical Pathology, The Canberra Hospital, Woden ACT2606, Australia. ⁵ School of Surgery and Pathology, University of Western Australia, Perth WA6909, Australia. ⁶ Core Clinical Pathology & Biochemistry, Royal Perth Hospital, Perth WA6001, Australia.

^aAddress correspondence to this author at: Department of Core Clinical Pathology and Biochemistry, Royal Perth Hospital, GPO Box X2213, Perth WA6847, Australia. Fax 61-8-9224-1789; e-mail samuel.vasikaran{at}health.wa.gov.au.

	Abstract

Top Abstract Introduction Materials and Methods Results Discussion References

 
Background: Clinical interpretation of laboratory results is an integral part of clinical chemistry. However, the performance goals for assessing interpretative commenting in this discipline have not been as well established as for the quality of analytical requirements.

Methods: We present a review of the 10 case reports circulated in the 2002 Patient Report Comments Program by the Royal College of Pathologists of Australasia (RCPA) and the Australasian Association of Clinical Biochemists Chemical Pathology Group of RCPA-Quality Assurance Programs Pty Ltd. Participants were expected to add an interpretative comment to a set of results accompanied by brief clinical details. Comments received were broken down into components that were translated into key phrases. An expert panel evaluated the appropriateness of the key phrases and proposed a suggested composite comment. A case summary/rationale was also returned to participants.

Results: There was considerable diversity in the range of interpretative comments received for each case report. Although the majority of comments received were felt to be acceptable by the expert panel, some comments were felt to be inappropriate, misleading, or in a few instances, dangerous.

Conclusion: The golden rule in medicine is "do no harm". Although there is no objective evidence that interpretive comments help to improve patient outcomes, if comments are added to reports it is important that they reflect accepted practice and current guidelines. It is of concern that a large proportion of comments returned were considered to be inappropriate and/or misleading. The Patient Report Comments Program has highlighted the need to consider limiting commenting to persons with clear expertise.

	Introduction

Top Abstract Introduction Materials and Methods Results Discussion References

 
Interpretative commenting is an important aspect of laboratory medicine (1), and most laboratories provide some form of interpretative comments on their laboratory reports. Although this is universal in anatomical pathology, not all clinical pathology reports have interpretative comments attached to them. The practice of attaching individualized comments to clinical biochemistry reports varies among countries and even among laboratories within a country. The utility of interpretative comments on laboratory results may also be dependent on the customer served, being potentially more useful for general (family) practitioners. Many of the standard comments on laboratory reports are generated by computer, whereas others are individualized depending on the results and the availability of clinical information.

The following factors favor the continuation, or even expansion, of the practice of adding comments to clinical pathology reports (2):

• Introduction of new and complex tests, including the availability of genetic testing, will increase the complexity of medical management. This may influence the interpretation and clinical applicability of existing tests.
  
• Increased electronic data communication will require clinicians to cope with this traffic and increase the desirability of interpretative filters.
  
• Because of competition among pathology laboratories, especially but not exclusively in the private sector, interpretative comments add value to the reports that customers receive.
  
• Given the increasing number of clinical and regulatory guidelines, the laboratory can provide guidance on appropriate test ordering, correct interpretation, and relevant follow-up of results.
  
• The capability of current computer networks has enabled the use of expert systems and the building of interpretative algorithms to provide relevant clinical interpretation.
  

Although the analytical activities in the clinical biochemistry laboratory are rigorously quality controlled and most pre- and postanalytical activities are monitored for errors, interpretative commenting in clinical chemistry has until recently escaped this scrutiny. Gordon Challand (3), through the Internet-based "Cases for Comments" program, highlighted the ability to objectively analyze and assess the quality of interpretative comments in clinical biochemistry results. The need for such formal assessment has been recognized by Clinical Pathology Accreditation Limited in the United Kingdom through their funding of a national external quality assessment scheme (NEQAS) ¹ for interpretative comments in clinical chemistry. There is now some degree of acceptance of quality assurance (QA) of interpretative commenting within the clinical biochemistry profession (4)(5). Recognizing these developments, the Royal College of Pathologists of Australasia/Australasian Association of Clinical Biochemists (RCPA-AACB) Chemical Pathology Group of RCPA Quality Assurance Programs (QAP) Pty Ltd initiated a quality assessment program in patient report commenting, which was piloted in 2000. This program has now gained acceptance among the chemical pathology profession in Australia as a tool for continuing professional development. The original design of the program, its assessment process, and a review of individual case reports have been discussed previously in the pilot program as well as in the subsequent year’s program (2)(6). Substantial changes have been made to the design of the program and method of analysis and feedback to participants as a result of the experience gained from previous cycles as well as regular feedback and review sessions with participants at scientific meetings. We present here the case reports distributed in the Patient Report Comments Program in 2002 and discuss the responses from and the feedback to the participants, the assessment of these responses by the program expert panel, and the format of the summary report.

	Materials and Methods

Top Abstract Introduction Materials and Methods Results Discussion References

 
Ten cases were distributed in 2002 to all participants of the Chemical Serum Chemistry program, which includes 600 laboratories. Two of these cases (cases 1 and 3) are shown in Table 1 , and the rest are available in the Data Supplement that accompanies the online version of this article at http://www.clinchem.org/content/vol50/issue1/. The 10 cases were selected from a routine clinical laboratory attached to a teaching hospital that also serves general practitioners as well as smaller district hospitals. The cases were representative of the spectrum of results routinely encountered in clinical biochemistry practice. With each of the case reports distributed, a standard format was followed with the provision of (a) age and gender of the patient, (b) location where the tests were ordered, (c) the set of biochemistry results for commenting, (d) clinical notes that would typically appear on the request form, and (e) additional relevant information, past or present, available to the laboratory. Space was provided to allow participants to write their interpretative comments.

Each laboratory could send an "official" response, which was generated by the pathologist or senior scientist who would customarily add comments on reports in routine practice. In addition, "unofficial" responses were also accepted from any other interested laboratory personnel because the program was aimed at individuals rather than laboratories. Computer-generated comments were accepted for submission, but these comments were not identified in the program. Currently, participation in this program attracts no subscription charges.

All comments received were broken down into components, and the components were translated/summarized into common keywords or phrases (Table 2 ). More than one key phrase could be generated from each submitted response. Responses were further subdivided as official or unofficial. An expert panel was selected that consisted of the Program Convenor, the chairman of the Chemical Pathology QAP Group, and the chief examiners of RCPA and AACB. One of the roles of the expert panel was to independently assess the appropriateness of the key phrases from all the comments received. The panel did not see the whole comments from individual participants, only the list of key phrases extracted from the comments submitted by all participants.

Each key phrase was classified as "preferred", "less relevant", or "unacceptable" by the expert panel (Table 2 ). The preferred key phrases were those judged to be appropriate for that report and that were useful to the attending clinician who would receive the report. Unacceptable comments were inappropriate and could potentially lead to mismanagement of patients. Comments that were classified as less relevant were seen as not useful to the requesting doctors and offering no guidance for patient management, although not erroneous or misleading. A summary report of each case was returned to the participants that included the classification of all key phrases, a frequency bar chart of the commonly used key phrases, a "suggested" comment as recommended by the expert panel, and a brief discussion of the case and one or two key literature references (cases 1 and 3 are shown in Table 3 and the rest of the cases are available in the online Data Supplement). No attempt was made to score the comments. A summary report of each case was returned to the participants within 2 weeks of receipt of their submission.

	Results

Top Abstract Introduction Materials and Methods Results Discussion References

 
The participation rate varied from case to case during the year (Table 4 ). The total number of key phrases extracted from comments of all participants varied depending on the spectrum and complexity of the case report. The mean number of key phrases for the 10 cases was 51 (range, 29–71; see Table 4 ). The break down into official and unofficial participants was, on average, relatively evenly distributed: 52% official and 48% unofficial (Table 4 ). Table 5 shows the total number of times the three categories of key phrases were used for each case by official and unofficial participants. Although the number of unacceptable key phrases for most cases was higher than the number of preferred key phrases, each unacceptable key phrase in general was used by only a few participants or, in many cases, by one participant only, whereas each preferred key phrase was used by a high proportion of participants. For example, for case 1, there were 8 preferred key phrases used a total of 200 times, whereas there were 12 unacceptable key phrases used a total of 19 times (Tables 4 and 5 ). The proportion of inappropriate key phrases in the interpretations submitted by the official and the unofficial participants were similar (Table 5 ).

The full list of key phrases used in comments received for case 1 and their classification according to appropriateness by the expert panel is shown in Table 2 . The most commonly used preferred key phrases were "suggest serum/urine osmolality" in 45 (46%) responses, "?secondary to medications" in 39 (40%) responses, "suggest urine electrolytes" in 34 (35%) responses, "severe hyponatremia" in 27 (28%) responses, "?hyponatremia cause of fit" in 16 (16%) responses, and "?psychogenic polydipsia" in 11 (11%) responses.

The full list of key phrases used in comments received for case report 3, which is an example of a more specialized case, are also shown in Table 2 . Sixty-two (75%) of the responses suggested "HFE genotype", and 50 (60%) raised the possibility of "?Hemochromatosis". These key phrases were classified as less relevant (7)(8). Key phrases such as "hemochromatosis unlikely" or "iron overload" were classified as unacceptable. Only 5 (6%) participants stated "diabetes not secondary to hemochromatosis", a preferred key phrase, in their comments. "Repeat iron studies", a preferred key phrase, was included in 24 (29%) of the responses, and in fact, this patient had returned a normal transferrin saturation on repeat testing.

	Discussion

Top Abstract Introduction Materials and Methods Results Discussion References

 
Interpretation of laboratory test results is a postanalytical activity. It requires an understanding of the analytical processes involved in generating the results, recognition of potential analytical and preanalytical errors, and correlation of results with the patient’s clinical status. Ideally, because no one individual can have full knowledge and expertise in all of the above aspects of the testing cycle, result interpretation should be a collaborative activity with input from the pathologist and the clinician. In reality this is not possible for every test result generated because of the sheer number of tests performed, nor is it necessary in the vast majority of cases. However, in a large proportion of results, attaching a comment adds value to the report and is appreciated by junior hospital doctors and general practitioners. Even specialists may be helped by interpretative comments, especially on test results outside their specialty.

The practice of attaching interpretative comments to clinical chemistry reports varies widely among countries and types of practices as well as the degree of specialization and complexity of the particular test. There are guidelines published by the Royal College of Pathologists in the UK for clinical authorization and reporting of results in clinical biochemistry laboratories (9). It is uncommon for individualized interpretative comments to be attached to general chemistry reports such as liver function tests, urea, and electrolyte profiles. This may be difficult because of the large number of such tests performed by a laboratory as well as the lack of clinical information available to the pathologist to make useful comments (10). However, recent developments in information technology mean that these hurdles are not insurmountable. The development of laboratory expert systems might help overcome the former problem, whereas availability of patient information by means of a unified information system may assist with the latter (11). Most laboratories have a roster of on-duty biochemists whereby pathologists and senior scientists offer interpretation over the phone or in person. The QA program described here may be directed at the activity of adding interpretative comments on printed reports as well as the provision of such verbal advice.

QA of interpretative commenting is still in its infancy, and the methods used need to be developed further and fine-tuned. What has become clear is that it is well accepted by the profession as a useful tool for continuing professional development in the countries in which it is being developed. The proponents hope that the development and administration of, and participation in, such programs would help the profession to scrutinize and improve the practice of interpretative commenting. The key lessons learned by the organizers of this program is that both participation and education could be enhanced by providing expert analysis of responses and an assessment of what is preferred and what is unacceptable (2). In addition, a rationale and key reference(s) are also now included in the summary report on the request of participants.

The analysis of comments into components and the translation into key phrases is subjective, and this step might introduce errors, thus contributing to the diversity and complexity of the interpretation. The key phrases are extracted from a sentence, but attempts were made to preserve the meaning and context of the interpretation. An independent review of this step has been introduced to improve consistency of key-phrase translation. In the NEQAS (UK) program for interpretative comments, each comment as a whole is scored instead of being translated into key phrases. We feel that classifying key phrases is more objective and of more educational value than scoring whole comments. The latter would also be more time-consuming to the panel. The panel of assessors in this program are "experts" holding designated positions in professional bodies, whereas the NEQAS program uses "peers". The assessor panel may not necessarily always get it right; e.g., in the assessment of one case, a key phrase classified as inappropriate appeared in the expert-panel’s suggested comment. The anomaly may be because the phrase is appropriate only if used in context (i.e., it is unacceptable if suggesting adequate treatment but preferred if suggesting possible overtreatment). However, an internal quality check has been introduced to confirm consistency between the suggested comment and the key phrase classification. The pros and cons of the various methods of analysis are at this stage unclear and need to be studied further.

Inappropriate commenting is an area of concern because the laboratories are seen as experts and are providing guidance to the requesting clinicians. Inappropriate comments appearing on patient reports with the frequencies listed in Table 5 could lead to a significant number of misdiagnoses and could be dangerous to patient care if acted on by the clinicians. The considerable number of inappropriate key phrases in the comments received could in part be explained by the various assumptions made by participants for cases in which sufficient clinical information was lacking. In reality, the pathologist would contact the requesting doctor by phone and obtain details of the case before commenting. The number of inappropriate key phrases listed may also relate to the fact that this is perceived as an educational program by chemical pathology trainees and junior scientists and that such comments may not in fact be used in real life. However, perhaps surprisingly, the distribution of inappropriate key phrases in the interpretations submitted by the official and unofficial participants were similar (Table 5 ), suggesting that, on average, professionals who offer interpretation of results in practice do not perform substantially better than other participants in this program. The number of inappropriate key phrases may also reflect recent changes in laboratories, which have led to a decrease in subspecialization with the merging of specialist laboratories into general chemistry laboratories.

These findings do indicate that interpretation provided by laboratory professionals with inadequate expertise can be clinically dangerous and highlight the need for improvement in the standards of interpretation currently provided. The profession needs to address the reasons for the current level of performance of the interpretative service it offers and take steps to greatly improve its quality if it is to contribute to the improvement of patient care and minimize medical errors. QA programs in interpretative commenting, such as that described here, may help in education and continuing professional development in this regard and in monitoring performance in the future.

	Footnotes

 
¹ Nonstandard abbreviations: NEQAS, national external quality assessment scheme; QA, quality assurance; RCPA, Royal College of Pathologists of Australasia; AACB, Australasian Association of Clinical Biochemists; and QAP, Quality Assurance Programs.

	References

Top Abstract Introduction Materials and Methods Results Discussion References

 

1. Fraser CG, Fogerty V. Interpreting laboratory results. BMJ 1989;298:1659-1660.
2. Vasikaran SD, Penberthy L, Gill J, Scott S, Sikaris KA. Review of a pilot quality assurance program for interpretative commenting. Ann Clin Biochem 2002;39:250-260.[Medline] [Order article via Infotrieve]
3. Challand GS. Assessing the quality of comments on reports: a retrospective study. Ann Clin Biochem 1999;36:316-322.
4. Marshall WJ, Challand GS. Provision of interpretative comments on biochemical report forms. Ann Clin Biochem 2000;37:758-763.
5. Sciacovelli L, Zardo L, Secchiero S, Zanintonotto M, Plebani M. Interpretive comments and reference ranges in EQA programs as a tool for improving laboratory appropriateness and effectiveness. Clin Chim Acta 2003;333:209-219.[Medline] [Order article via Infotrieve]
6. Lim EM, Vasikaran SD, Gill J, Calleja J, Hickman PE, Beilby J, et al. A discussion of cases in the 2001 RCPA-AQAP Chemical Pathology Case Report Comments Program. Pathology 2003;35:145-150.[Medline] [Order article via Infotrieve]
7. Vautier G, Murray M, Olynyk JK. Hereditary haemochromatosis: detection and management. Med J Aust 2001;175:418-421.[Medline] [Order article via Infotrieve]
8. Bulaj ZJ, Ajioka RS, Phillips JD, LaSalle BA, Jorde LB, Griffen LM, et al. Disease-related conditions in relatives of patients with hemochromatosis. N Engl J Med 2002;343:1529-1535.
9. . Standing Advisory Committee in Chemical Pathology. Guidelines for the provision of interpretative comments on biochemical reports. Bull R Coll Pathol 1998;104:25.
10. Prinsloo PJ, Gray TA. A survey of laboratory practice in the clinical authorization and reporting of results. Ann Clin Biochem 2003;40:149-155.[Medline] [Order article via Infotrieve]
11. Smith BJ, McNeely MD. The influence of an expert system for test ordering and interpretation on laboratory investigations. Clin Chem 1999;45:1168-1175.[Abstract/Free Full Text]

The following articles in journals at HighWire Press have cited this article:

				D Sinclair, A Spedding, and R Young Can the laboratory affect the investigation and diagnosis of primary biliary cirrhosis? J. Clin. Pathol., April 1, 2006; 59(4): 360 - 362. [Abstract] [Full Text] [PDF]

				S. Skeie, C. Perich, C. Ricos, A. Araczki, A. R. Horvath, W. P. Oosterhuis, T. Bubner, G. Nordin, R. Delport, G. Thue, et al. Postanalytical External Quality Assessment of Blood Glucose and Hemoglobin A1c: An International Survey Clin. Chem., July 1, 2005; 51(7): 1145 - 1153. [Abstract] [Full Text] [PDF]

				I S Young Interpretive comments on clinical biochemistry reports J. Clin. Pathol., June 1, 2005; 58(6): 575 - 575. [Full Text] [PDF]

				G. Eisenhofer, J. W.M. Lenders, D. S. Goldstein, M. Mannelli, G. Csako, M. M. Walther, F. M. Brouwers, and K. Pacak Pheochromocytoma Catecholamine Phenotypes and Prediction of Tumor Size and Location by Use of Plasma Free Metanephrines Clin. Chem., April 1, 2005; 51(4): 735 - 744. [Abstract] [Full Text] [PDF]

				M. Laposata Patient-specific Narrative Interpretations of Complex Clinical Laboratory Evaluations: Who Is Competent to Provide Them? Clin. Chem., March 1, 2004; 50(3): 471 - 472. [Full Text] [PDF]

eLetters:

This Article Abstract Full Text (PDF) Data Supplement Submit an electronic Letter to the Editor about this paper View responses Alert me when this article is cited Alert me when eLetters are posted Alert me if a correction is posted Citation Map Services Email this article to a friend Similar articles in this journal Similar articles in ISI Web of Science Similar articles in PubMed Alert me to new issues of the journal Download to citation manager Citing Articles Citing Articles via HighWire Citing Articles via ISI Web of Science (17) Citing Articles via Google Scholar Google Scholar Articles by Lim, E. M. Articles by Vasikaran, S. D. Search for Related Content PubMed PubMed Citation Articles by Lim, E. M. Articles by Vasikaran, S. D. Related Collections Laboratory Management