| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
|
|
|
|||||||
|
|||||||||
Articles |
 |
Abstract |
|---|
 Top Abstract IntroductionReferences |
|---|
Key Words: indexing terms: quality control • point of care testing
|
Introduction |
|---|
|
TopAbstractIntroductionReferences |
|---|
Much of what hospitals do today they do because of an unquestioned obedience to outmoded notions about how to get work done. (Hospitals aren't alone, of course.) Modern farmers use leading-edge tractors to plant corn in rows 40 inches apart and probably can't tell you why. The answer? That's the width necessary to accommodate a plow horse.
Change in any environment can be difficult. Change in the delivery of healthcare is particularly difficult because lives are attached to the outcomes, and the consequences of errors can be life-threatening. This concern, coupled with the habits of historical clinical/medical practice, has precluded many changes from taking place in the clinical laboratory. With the availability of unit-use tests, however, the laboratorian's role is changing and must be reevaluated (2). One subject that has acquired a high profile is that of quality control (QC) for unit-use devices.1
History provides numerous examples of process controlls applied to
unit-use or single-test systems. Perhaps the most common is that of the
DuPont (Wilmington, DE) aca® analyzer in the
1960s, followed by the Ektachem® (Johnson & Johnson,
Rochester, NY) system. A nonautomated example of unitized testing
methods is the use of commercially prepared microbiology blood agar
plates, which are, by definition, unit-use systems. Performing QC on
each plate before use would require contaminating each plate, thereby
making it unusable for any patient's sample. In 1984, the National
Committee on Clinical Laboratory Standards (NCCLS) formed the
Subcommittee on Media Quality Control to specify the requirements for
quality assurance (QA) of culture media. This work resulted in the
publication of M-22, "Quality assurance for commercially prepared
microbiological culture media." The proposed standard was published
in 1985, the approved standard in 1990, and the second edition of the
approved standard in December 1996. The basic premise of M-22 is that
retesting of commercially prepared media imposes a substantial
financial burden on the clinical microbiology laboratory that might not
be necessary for those media of proven reliability (3).
The Subcommittee was able to gauge reliability by referring to College
of American Pathologists (CAP) surveys (4). From a
Microbiology Proficiency Testing Survey representing >350 000 lots
and accounting for 67 x 106 plates, tubes, or bottles, the
Subcommittee set a failure rate of 
0.3% as being acceptable. That
is, users of media plates that meet this level of failure, as
documented by the manufacturer, do not have to retest.
A Unit Use Subcommittee of NCCLS has been formed and has discussed using existing examples of QA/QC for other tests or devices; the plated media example is being considered. However, the difficulty of making a transition from media to unit use is in the numbers. The CAP Proficiency Study numbers are huge, whereas most unitized tests currently available (except glucose) do not begin to approach this test volume. Unit-use glucose testing is waived in the CLIA '88 regulations and therefore already is not subject to the "two level per day of use" rule. Nevertheless, this model has provided some guidance in the Subcommittee's discussions.
QC procedures were first widely used in the late 1950s and early 1960s, when it became necessary to check the instrument in use as well as the analytical processes. One of the first devices to demonstrate the need for QC was the AutoAnalyzer® (the SMA6/60; Miles, Terrytown, NY). QC was recommended for each batch, defined as the number of patients' samples to be processed at one time. After many years and a great deal of discussion, a "run" began to be defined by time, i.e., the number of hours—either an 8-h or a 24-h shift.
James Westgard of the University of Wisconsin, well known as developer of the Westgard multirules, has described the evolution of laboratory statistical QC practices (5). As he points out, in the early 1960s and '70s the first generation of QC was put into general practice primarily to promote the fact that laboratories were doing QC. The primary tracking method was the Levey–Jennings chart for graphing the analytical results. Westgard's multirules, which aimed at utilizing the Levey–Jennings charts to maximize error detection and minimize false rejections, ushered in the second generation of QC in the 1980s. The third generation, Westgard suggests, could be QC designed for individualized tests to assure detection of medically important errors at the lowest cost (6)—in other words, what is the allowable error? However, this practice has not been widely used in the '90s.
Because of Westgard's long-term leadership in QC and his reputation in this area, he was asked to join the NCCLS Subcommittee on Unit Use Testing and, at the Subcommittee's first meeting, he gave the preceding historical review of QC. He also noted that, even as change was taking place in QC procedures, updates and innovations by in vitro diagnostic (IVD) manufacturers were leading to considerable changes in the test instruments themselves. These innovations continued through four generations of IVD devices, i.e., manual methods, continuous flow, multitest parallel continuous flow, and highly precise/highly stable instrumentation. We now await the fifth generation. According to Westgard, however, QC laboratory statistical models stalled somewhere after the second generation. Therefore, he concluded, the NCCLS Subcommittee must choose to either move QC development through three generations very rapidly or create an entirely new paradigm. The Subcommittee, while not making a formal choice, is developing a guideline to assist users, manufacturers, and regulators in dealing with QC issues. Whether this will be a new paradigm or a leap frog of the QC generations remains to be seen.
The Subcommittee for Unit Use Testing held its first meeting on March 29, 1996. One of the larger NCCLS subcommittees, it is made up of clinical laboratory professionals as well as representatives from regulators, accrediting agencies, and manufacturers.2 The concept for the Subcommittee evolved from two perspectives. The first was the desire to develop guidelines and standards in a collaborative rather than adversarial manner. Historically, IVD manufacturers have addressed QC in the design or labeling of their products but leave it to the users to implement QC—either adopting the manufacturer's system or developing their own. Neither user nor manufacturer worked with the accrediting agencies. It seems logical, however, that the manufacturer, the user, and the regulator should work together to agree on how to produce quality testing with available resources. The manufacturer would then build the designated system into the product, the user would use it accordingly, and the regulator would inspect to verify the proper implementation of the process. This is a prospective, quality improvement process as opposed to a retrospective, punitive process.
The second perspective was a financial one. The resources required to meet the regulations added cost but not necessarily value and were therefore potentially prohibitive to innovation. This is not dissimilar to the catalyst for M-22, the plated media guideline. If unit-use test devices were to have an opportunity to be implemented, QC requirements and their cost would have to be addressed. An example of the financial issues involved was provided by Paula Santrach, Mayo Clinic, for activated clotting times (7). Reviewing the QC material and number of employees necessary to meet the accrediting agency's requirements, Santrach discovered that her laboratory would spend >$289 000 annually and 3.9 full-time (employee) equivalents (FTEs) for reagents and control materials for 25 instruments measuring 43 channels to meet the requirement of "two levels per day of use" for 7 days a week. These data are based on ~$3.00 per QC test and 4 min per test. The basic formula for determining the FTE number was the number of channels x the number of control levels x the number of shifts x the number of days x 4 min per test. The number of QC tests per day was thus 258 and, at 4 min per test, would require >17 h each day. The calculation does not include the effects of logistics, but consider how long QC would take for 25 devices located in presumably 25 different locations perhaps several minutes apart. And finally, no increased accuracy, precision, or improved reliability was expected from this additional testing.
At the first Subcommittee meeting, the following guideline objective was developed:
To: Develop a guideline for establishing a quality system for unit-use test devices Which is: Practical to implement Applicable to various devices and settings; and Scientifically based So that: Device manufacturers, users, and regulatory and accrediting agencies can assure that correct results are obtained and that sources of error are identified, understood, and managed.
Other points to be addressed were identified, including:
1) A working definition for categorizing a test system as unit use: A system is a unit-use system if the container where the test is performed is always discarded after each test, and the reagents, calibrators, and wash solutions are typically segregated as one test (i.e., there is no interaction of reagents, calibrators, and wash solutions from test to test).
2) The components of the testing process, defined as: specimen collection, sample presentation, instrument/reagents, results/readout/raw data, preliminary review, integration/report in patient's chart, operator, and testing environment.
3) Sources of error.
4) Management techniques for addressing the sources of error.
The Subcommittee felt that it was imperative to broaden its thinking about the testing process, to cover more steps within the process rather than only the quality of the instrument and reagents. Therefore, the testing process (no. 2 above) was defined to include both preanalytical and postanalytical steps.
According to the objective, the guideline is to apply to all testing systems. This flexibility can be provided only within a framework that permits innovation. Therefore, one focus of the Subcommittee will be to identify all potential sources of error in the testing process. The Subcommittee plans to identify as many sources of error as possible and use the list as a "points to consider" document for both manufacturers and operators. The guideline will probably recommend that management of the potential sources of error be addressed by each manufacturer in relation to a specific system. The manufacturer will try to document to the users'—and ultimately, the regulators'—satisfaction that the source of error has been eliminated or controlled. If this is not possible, the user—with or without the manufacturer's assistance—will be charged with managing the source of error. This approach will permit development of a quality system that includes review of all testing areas, identifies the potential sources of errors, and manages errors. The distinction is an important one. The Subcommittee understands that, as a practical matter, not all errors can be eliminated but all can be identified and managed appropriately.
Another area of discussion within the Subcommittee involves the quality system methods and processes used in manufacturing. Beckham (1) noted many parallels between manufacturing and healthcare, even though hospitals (and by extension, laboratories) "don't appreciate their lineage to manufacturing. Few hospitals would describe themselves as being in the business of manufacturing care. Yet, the way they produce care suggests they are very much creatures of the manufacturing jungle."
The Subcommittee's discussions have uncovered many aspects of quality manufacturing that could potentially be utilized in the "manufacturing" of IVD results, particularly in a unit-use format. These methods and processes, or Good Manufacturing Processes, are utilized routinely by manufacturers and by the Food and Drug Administration (FDA) as part of their on-site manufacturing inspections. Coincidentally, the FDA has just published the new Quality Systems Regulations [8], a revision to the Good Manufacturing Practices (published in 1990) to now include preproduction and design controls. FDA inspections will gradually include this regulation beginning in June 1997, with full implementation in June 1998. The product life-cycle, as defined by FDA, now includes product development, manufacturing, product handling (packaging, storage, and shipment), and product use. The Subcommittee believes that a comprehensive quality system that builds on existing methods and identifies potential sources of error in each area will allow believable but erroneous test results to be identified and managed. This management component will enable innovation in both manufacturing and implementation and provide the necessary oversight and control of unit-use test systems.
One such management technique is the failure mode and effects analysis
(FMEA), a systematic approach used by manufacturing engineers to
identify potential failure modes and their effects
(9). These failures are broken down into
systems, designs, and processes. Process FMEA, for example, scores the
effect of a failure (error) in three ways: detection, occurrence, and
severity (Table 1
). The error or failure is graded by the ease of detection, the
frequency of occurrence, and the severity of the effect in the event of
failure. Each aspect is scored from 1 to 10, and all are combined to
determine the risk priority number. The higher the number, the more
urgent it is to allocate resources toward correcting the problem. FMEA
is not new to the manufacturers on the Subcommittee but is new to most
of the other members and advisors and has not previously been applied
to IVDs. Therefore, the Subcommittee is carefully investigating this
and looking for the best way to apply the logic. One area under
discussion would be the adding of a fourth category to FEMA: clinical
impact. This would allow the laboratory to set limits around the
criticality of a test, e.g., cholesterol screening vs determination of
cardiac markers to aid a decision to administer a thrombolytic agent to
an emergency room patient with a potential myocardial infarction.
|
The Subcommittee hopes to have a tentative guideline out for comment during the second quarter of 1997. As indicated by the discussion above, the guideline will consider many issues with significant dynamics. Many recommendations may involve dramatic changes for both manufacturers and users, especially because QA/QC processes have not evolved over time. Although the changes required may appear abrupt, they are necessary and, if done properly, will both add value and reduce overall costs.
|
Footnotes |
|---|
1 Nonstandard abbreviations: QC, quality control; QA, quality assurance; IVD, in vitro diagnostics; FTE, full-time equivalent; and FMEA, failure mode and effects analysis. 
2 NCCLS Subcommittee on Unit Use Testing: David L. Phillips (Chair), Boehringer Mannheim Corp.; Paula J. Santrach, M.D. (Vice-Chair), Mayo Clinic; Richard Anderson, Ph.D., Biosite Diagnostics; Anne Belanger, JCAHO; Rosemary Bakes-Martin, M.T. (ASCP), M.S., Centers for Disease Control and Prevention; Cecelia S. Hinkel, M.T. (ASCP), Health Care Financing Administration; Wendell R. O'Neal, Ph.D., The WHISK Group; Nina Peled, Ph.D., i-STAT Corp.; Cornelia B. Rooks, M.A., FDA; Walter L. Sembrowich, Ph.D., Diametrics Medical, Inc.; James O. Westgard, Ph.D., University of Wisconsin; and Ronald J. Whitley, Ph.D., University of Kentucky.
|
References |
|---|
|
TopAbstractIntroductionReferences |
|---|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
