Jacoline Bouvy won the Night of Descartes Essay Contest on Evidence Based Medicine, featured in the latest UBlad:
RANKING EVIDENCE IS NOT ENOUGH
Doctors have always treated patients according to what they believed were effective treatments. Until the late 19th century, however, bloodletting was thought to be a cure for a variety of diseases. Today, we prefer treatments that are ‘scientifically proven’ to be effective and expect our doctors to practice evidence-based medicine. But evidence-based medicine is not a medical doctrine: it is a set of tools that allows doctors to practice medicine by preferably using treatments that have been shown to be efficacious and effective. It helps doctors to be able to provide the ‘best’ therapies to their patients. However, evidence-based medicine is far from perfect and holds several weaknesses.The foundation of evidence-based medicine is the ‘hierarchy of evidence’ model, which ranks different types of scientific research study designs according to the authority these types of research hold. The hierarchy of evidence model wrongly regards the randomized controlled trial (RCT) (and systematic reviews of RCTs) as the ‘best’ type of medical research. Furthermore, it discriminates against lower ranked study designs that in the past have contributed to a great extent to current knowledge of diseases.In the development of medical evidence-based guidelines, the hierarchy of evidence model is used to rank the authority of different types of research. The randomized controlled trial and systematic reviews of RCTs are ranked above observational studies, case reports and expert opinion. A systematic review of at least two independent randomized double-blinded controlled trials of sufficient quality and size is considered the highest form of evidence in medicine followed by a single randomized double-blinded controlled trial. Ranked below RCTs are comparative studies (e.g. case-control studies and open trials), and at the bottom we find non-comparative observational studies and expert opinion.The randomized controlled trial is the closest medicine has come to a formal experiment which is why it is ranked highest. Yet, it is exactly the nature of RCT design that is problematic for its application in medicine: the highly controlled circumstances and selected patient population result in problems with external validity. The homogenous patient sample of an RCT is not very representative for the real-life, daily practice of health care, where the patient population is naturally very heterogeneous. Possibly even more important, pharmaceutical companies that have an obvious interest in publishing promising results of their products often fund RCTs. This does not mean we cannot trust the outcomes of their studies; it does mean we have to be aware of the issue of objectivity in their studies. Ideally, their results should be repeated by independent studies.
Unfortunately, neither does independent research guarantee objectivity of study outcomes. The well-known ‘Scientists behaving badly’ publication (2005) has shown that 33 percent of researchers have engaged in at least one form of serious misbehavior in research (Martinson et al. 2005*). Thus, objectivity in medical research and publications in scientific journals does not exist. The hierarchy of evidence model does suggest that when study design conditions (e.g. sufficient quality and size) are met, RCTs will always produce reliable results. This is a wrong assumption.Randomized controlled trials have another major fallacy: they are short-term oriented. Observational studies (that are ranked below RCTs) are essential in investigating long-term effects of treatments and determinants of disease. The Framingham Heart study that started in 1948 is an example of a well-known observational study that has contributed greatly to our knowledge of cardiovascular diseases. Also, the relationship between smoking and lung cancer was shown not by randomized controlled trials but by epidemiological study designs. These examples clearly show that study design does not necessarily indicate the importance of the study’s results to the field of medicine.Evidence-based medicine and its hierarchy of evidence model clearly put randomized controlled trials above observational studies and, therefore, forces rigid judgment on their value and utility in medicine. But history has shown us that we need to look beyond study design to determine the true quality of scientific research and the extent to which published studies contribute to our knowledge of diseases and treatments. The hierarchy of evidence model is useful but has several limitations that create a need to look beyond evidence-based medicine. True evidence-based medicine should value research according to its importance and contribution to our increasing understanding of medicine. The evidence-based medicine model that simply ranks scientific studies based on their study design is not sufficient.
*Martinson BC, Anderson MS, de Vries R. Scientists behaving badly. Nature 2005;435(June):737-738.