Critics of EBM say lack of evidence and lack of benefit are not the same, and that the more data are pooled and aggregated, the more difficult it is to compare the patients in the studies with the patient in front of the doctor — that is, EBM applies to populations, not necessarily to individuals. In The limits of evidence-based medicine,[2]Tonelli argues that "the knowledge gained from clinical research does not directly answer the primary clinical question of what is best for the patient at hand." Tonelli suggests that proponents of evidence-based medicine discount the value of clinical experience.
However, many proponents of EBM argue that the best practice of EBM does not discount clinicians' own experience. For instance David Sackett writes that "the practice of evidence based medicine means integrating individual clinical expertise with the best available external clinical evidence from systematic research".[23]
Although evidence-based medicine is becoming regarded as the "gold standard" for clinical practice and treatment guidelines,[citation needed] there are a number of reasons why most current medical and surgical practices do not have a strong literature base supporting them.
- In some cases, such as in open-heart surgery, conducting randomized, placebo-controlled trials would be unethical, although observational studies may address these problems to some degree.
- Certain groups have been historically under-researched (racial minorities and people with many co-morbid diseases), and thus the literature is sparse in areas that do not allow for generalizing.[24]
- The types of trials considered "gold standard" (i.e. randomized double-blind placebo-controlled trials) may be expensive, so that funding sources play a role in what gets investigated. For example, public authorities may tend to fund preventive medicine studies to improve public health as a whole, while pharmaceutical companies fund studies intended to demonstrate the efficacy and safety of particular drugs.
- The studies that are published in medical journals may not be representative of all the studies that are completed on a given topic (published and unpublished) or may be misleading due to conflicts of interest (i.e. publication bias).[25] Thus the array of evidence available on particular therapies may not be well-represented in the literature. A 2004 statement by the International Committee of Medical Journal Editors that they will refuse to publish clinical trial results if the trial was not recorded publicly at its outset, may help with this, although this has to date still not been actioned.
- The quality of studies performed varies, making it difficult to generalize about the results.
An additional problem is that large randomized controlled trials are useful for examining discrete interventions for carefully defined medical conditions. The more complex the patient population (e.g. severity of condition, co-morbid conditions, etc) in the study, the more difficult it is to assess the treatment effect (i.e., treatment mean - control group mean), relative to the random variation (within group variation of both the treatment and control groups). Because of this, a number of studies obtain non-significant results, either because there is insufficient power to show a difference, or because the groups are not well-enough "controlled". Ironically, the fewer restrictions there are on who can participate in a study (i.e., the greater the generalizability of the results to the type of patient being seen in a real world setting) the less able the study to detect real differences between groups for a given sample size.
Furthermore, evidence-based guidelines do not remove the problem of extrapolation to different populations or longer timeframes. Even if several top-quality studies are available, questions always remain about how far, and to which populations, their results are "generalizable". Furthermore, skepticism about results may always be extended to areas not explicitly covered: for example a drug may influence a "secondary endpoint" such as test result (blood pressure, glucose, or cholesterol levels) without having the power to show that it decreases overall mortality or morbidity in a population.
In managed healthcare systems, evidence-based guideline have been used as a basis for denying insurance coverage for some treatments which are held by the physicians involved to be effective, but of which randomized controlled trials have not yet been published. In some cases, these denials were based upon questions of induction and efficacy as discussed above. For example, if an older generic statin drug has been shown to reduce mortality, is this enough evidence for use of a much more expensive newer statin drug which lowers cholesterol more effectively, but for which mortality reductions have not had time enough to be shown?[26] If a new, costly therapy that works on tumor blood vessels causes two kinds of cancer to go into remission, is it justified as an expense in a third kind of cancer, before this has specifically been proven?[27]. Kaiser Permanente did not change its methods of evaluating whether or not new therapies were too "experimental" to be covered, until it was successfully sued twice: once for delaying IVF treatments for two years after the courts determined that scientific evidence of efficacy and safety had reached the "reasonable" stage, and in another case where Kaiser refused to pay for liver transplantation in infants when it had already been shown to be effective in adults, on the basis that use in infants was still "experimental."[28] Here again the problem of induction plays a key role in arguments.
