| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
|
|
|
|||||||
|
|||||||||
Editorials |
1 Departments of Laboratory Medicine and Pathology , Children’s Hospital and Harvard Medical School, Boston, MA
2 Centre for Statistics in Medicine, Oxford, UK
3 Academic Medical Center, University of Amsterdam, Amsterdam, The Netherlands
aAddress correspondence to this author at:, Department of Laboratory Medicine, Children’s Hospital Boston, 300 Longwood Ave., Boston, MA 02115, E-mail nader.rifai{at}childrens.harvard.edu
Truth is not only violated by falsehood; it may be equally outraged by silence. —Henri-Frédéric Amiel (1812–1881)
Publication bias and selective reporting are recognized problems in the biomedical literature (1)(2)(3). Positive or favorable findings are more likely to be reported than negative or inconclusive ones. Authors are often disappointed by negative findings and use multiple comparisons and subgroup analyses to generate positive results, while giving scant attention to analyses of negative results (4). Contrary to common beliefs, studies with positive results and those with negative results have been shown to be equivalent with respect to the quality of study design and execution (5), and authors and sponsors, rather than journal editors, appear to be primarily responsible for the reporting bias (5)(6)(7). Publication of only selected results distorts the research evidence base that underpins the practice of evidence-based medicine. This practice diminishes the value of systematic reviews and metaanalyses, leads to unnecessary duplication of efforts, and prevents the derivation of unbiased estimates for the risk–benefit ratio of a drug therapy or a diagnostic evaluation. Furthermore, such dubious publication practices are inconsistent with researchers’ moral responsibilities (8).
Concerns over selective reporting of clinical trial results have been growing over the past several years (9)(10)(11). Comprehensive registration of clinical trials and public disclosure of study results have been suggested as the optimal solutions for this problem. Efforts to address this issue were intensified after the New York State Attorney General filed a lawsuit against GlaxoSmithKline on June 2, 2004, alleging the withholding of information regarding the efficacy and safety of paroxetine, a selective serotonin reuptake inhibitor, in children with depression (12). Shortly thereafter, the House of Delegates of the American Medical Association called on the US Department of Health and Human Services to establish a comprehensive national registry. Furthermore, the International Committee of Medical Journal Editors (ICMJE)1 published a statement in major medical journals requiring that starting July 1, 2005, all clinical trials must be registered as a condition to be considered for publication (13). Of all available registries at the time, the ICMJE recognized onlyhttp://www.clinicaltrials.org , which was authorized by the FDA Modernization Act of 1997 and sponsored by the US National Library of Medicine, as a registry that fulfilled the required criteria. Although the ICMJE establishes policy only for its 12 member journals (see http://www.icmje.org for a detailed description of the ICMJE and its purpose), the number of registered clinical trials increased suddenly by 67% after implementation of the ICMJE policy, from 15 000 to 25 000 (14). As of March 2008, approximately 52 000 clinical trials have been registered on this site.
Although attention has been focused on clinical trials, selective publishing has also been reported as a problem in observational and laboratory-based investigations (4)(5)(6). In fact, reporting bias has been greater in observational and laboratory-based experimental studies than in randomized clinical trials (5). Unfortunately, no means are currently available to determine the real extent of this problem in diagnostic and prognostic studies, and such assessment is urgently needed (3)(15).
While the problem of reporting bias was being addressed by medical organizations and journals, the clinical laboratory community was silently watching. We believe that it is time to act. It is imperative to create whatever measures are necessary to achieve the transparency in reporting of diagnostic and prognostic studies that the scientific community, the patients we serve, and the general public expect of our profession. The road map devised by our colleagues to tackle this issue for clinical trials can be used as a guide. We believe that only with the collaborative involvement of authors, sponsors, reviewers, journal editors, scientific professional organizations, and federal agencies can this issue be adequately addressed. We propose the following actions:
(1) The scientific and medical communities should move away from the harmful notion that studies can usefully be classified as "positive" or "negative" in relation to the statistical significance of the results. Such terms cause much confusion, and indeed harm, through this bias that is introduced into the published literature.
(2) Authors should recognize that "negative" results from well-conducted studies are as important to report as "positive" or "favorable" ones. A retrospective survey of 487 research projects approved by the Central Oxford Research Ethics Committee and conducted over a 4-year period showed that the quality of design and execution for positive and negative studies were equivalent (5). Furthermore, both positive and negative findings have been shown to be equally reported in multicenter trials and government-funded studies (5). Such reports are essential for the assessment of the efficacy of a drug or laboratory test and have the potential to change the practice of medicine. As indicated earlier, authors and their sponsors seem to bear primary responsibility for the reporting bias. Only 3%–9% of manuscripts with negative or inconclusive findings were reported to have been rejected for publication by journal editors (5)(6). Another disturbing reason cited for the failure to publish negative findings was sponsor control of the data and refusal to supply the data to the investigators (5)(16). It is the responsibility of research centers to protect their own investigators from overly aggressive restriction by sponsors.
(3) Reviewers should regard the quality of the study design and its execution as having greater importance than the actual results. They should check whether authors have complied with study design–specific reporting guidelines, such as CONSORT, STARD, and REMARK (see http://www.equator-network.org). Recent reports in several journals, including this one, have demonstrated that although STARD has been a standard for reporting diagnostic-accuracy studies since 2003, there is still much room for improvement in bringing articles into conformance with these important guidelines (17)(18). The emphasis on a P level of statistical significance of <0.05 should certainly be questioned, and an emphasis on meaningful estimates and confidence intervals should be encouraged. Reviewers should be particularly sensitive to the practice of "data dredging," in which often-meaningless subanalyses and comparisons are carried out to unearth positive associations when the main outcome of the study was found to be negative or inconclusive. In addition to being scientifically flawed, such an approach can be misleading, and to present such analyses without disclosing the extent of such post hoc analyses is dishonest. As the technologies for new diagnostic tests become more complex, it also becomes increasingly important to recognize the need to establish test cutoff points and performance targets in preliminary training sets and then to establish final performance in independent clinically relevant testing sets.
(4) The principal means to ensure transparency and reduce (or hopefully eliminate) publication bias would be to have a comprehensive registry of projects that is operated by a nonprofit organization, is electronically based, is easily searchable, and is freely accessible to scientists, patients, and the general public. Such a registry would not be intended to replace regulatory oversight by either the US Food and Drug Administration or the Centers for Medicare and Medicaid under the Clinical Laboratory Improvement Amendments. Rather, it is intended to provide a single source that would help inform the laboratory community about the latest findings of ongoing research. Study findings could be added when they become available. Unfortunately, no such registry currently exists for diagnostic and prognostic studies. As of April 2007, the ICMJE recognizes 5 trial registries in 4 countries as meeting the required criteria (14). Clearly, there is no need to establish a registry de novo for diagnostic and prognostic studies, but there is a need to explore the possibility of such a registry being hosted on one of the existing and ICMJE-recognized sites. These efforts should be led by laboratory-based professional societies, such as the American Association for Clinical Chemistry, the College of Pathologists, the American Society of Clinical Pathologists, the American Society for Microbiology, and these organizations’ international counterparts, as well as appropriate governmental agencies. These organizations share common goals and similar vested interests; they strive to promote high standards for laboratory medicine and to safeguard the public from suboptimal medical practices. Working toward the development or identification of a suitable registry is, therefore, a goal that is consistent with their missions and objectives. The FDA Amendments Act of 2007 (FDAAA) (19) includes new requirements for drug, biological, and device companies to begin reporting study design information and eventually the results of certain clinical trials to the http://www.clinicaltrials.gov Web site. The Food and Drug Administration and the NIH are currently working to determine how this new program will be implemented (9).
(5) Once a registry is identified, the editors of the major journals in laboratory medicine should follow the lead of the ICMJE and require that diagnostic and prognostic studies be registered ab initio as a condition for consideration for publication.
Publication bias and selective reporting are serious concerns that can undermine the trust of the general public in our profession, prevent us from assessing the true value of our services, and hinder our ability to deliver the best and most cost-efficient healthcare to our patients. It is imperative that we establish means to assess the actual magnitude of this problem in our discipline and then address it properly and judiciously. Although exact figures are currently unknown, it is reasonable to believe that diagnostic and prognostic studies fall prey to considerable publication bias. We sincerely hope that our editorial highlights this issue and provides the impetus needed to start a dialog between scientific professional organizations, journal editors, and government agencies.
Acknowledgments
Grant/Funding Support: None declared.
Financial Disclosures: None declared.
Acknowledgments: We are grateful to Steve Gutman for his thoughtful comments and review of this manuscript.
Footnotes
1 Nonstandard abbreviation: ICMJE, International Committee of Medical Journal Editors. 
References
The following articles in journals at HighWire Press have cited this article:
|
|
 |
|
  H. Hemingway, R. D Riley, and D. G Altman Ten steps towards improving prognosis research BMJ, December 30, 2009; 339(dec30_1): b4184 - b4184. [Full Text]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
