“Evidence-Based Psychotherapy” versus “Scientifically Oriented Psychotherapy”
I just stumbled across a new paper by David and Montgomery (2011), who provide a novel system for categorizing psychotherapies in terms of their quality of evidence. One reason we named this blog Science-Based Psychotherapy, is to highlight some of the flaws in the current methods of evaluating evidence-based practice. I hope that some of the recommendations of David and Montgomery (2011) get adopted, because their guidelines would be a huge advance over the current state of affairs. As stated in the article:
…all the current systems of evaluating evidence-based psychotherapies have a significant weakness; they restrict their focus on evidence to data supporting (psycho)therapeutic packages while ignoring whether any evidence exists to support the proposed theoretical underpinnings of these techniques. (i.e., theory about psychological mechanisms of change; p. 90)
Evidence-based therapy lists ignore basic science and theory
One big problem of the current methods of evaluating evidence is the lack of attention to basic science and theory. The result is that therapy packages that are based on theories that have been clearly invalidated can still appear to be scientifically credible:
By ignoring the theory, the evaluative frameworks of various health-related interventions (including psychotherapy), technically (a) allow pseudoscientific (i.e., ‘‘junk-science’’) interventions to enter into the classification schemes and or (b) bias the scientific research in a dangerous direction (p. 90).
The danger of these kinds of incentives is that they push researchers to focus solely on outcome research at the expense of testing and refining the scientific theories that will allow for future advances in therapy.
…a consequence of current classification schemes (which consistently do not address underlying theories about mechanisms of change) is that as long as there are randomized trial data, the validity of the underlying theory is less relevant (p. 90).
The current evaluative systems focus on only one kind of evidence: outcome evidence based on the performance of particular therapy packages. This evidence is typically in the form of randomized controlled trials (RCTs). What David et al. add is a second factor that focuses on evidence for the underlying theory.
They propose that each of these two factors are evaluated on three levels
a) empirically well supported;
(b) equivocal no clear data, which includes–not yet evaluated, preliminary data, or mixed data
(c) strong contradictory evidence (SCE; i.e., invalidating evidence).
Here’s their diagram showing how this breaks down:
One of the cool things about this framework is that it allows distinctions between therapies with both types of evidence and therapies that only have one form of evidence. They call those therapies with the highest levels of evidence “Scientifically Oriented Psychotherapies.”
Scientically oriented psychotherapies (SOPs) are those which do not have clear SCE for theory and package; the highest level of validation of a SOP is that in which both the theory about psychological mechanisms of change and the therapeutic package are well validated (i.e., Category I). A SOP seeks to investigate empirically both the therapeutic package in question and the underlying theory guiding the design and implementation of the therapeutic package (i.e., theory about mechanisms of change; p. 91).
A definition of pseudoscience
This allows for a pretty solid definition of a therapy based on pseudoscience.
Pseudoscientically oriented psychotherapies (POPs) are those that claim to be scientific, or that are made to appear scientific, but that do not adhere to an appropriate scientific methodology (e.g., there is an overreliance on anecdotal evidence and testimonial rather than empirical evidence collected in controlled studies; Lilienfeld, Lynn, & Lohr, 2003)…. We define POPs as therapies used and promoted in the clinical field as if they were scientifically based, despite strong contrary evidence related to at least one of their two components (i.e., therapeutic package and theory; p. 92).
One consequence of this approach is that it allows for the identification of therapies that have accumulated evidence of effectiveness, but where the theory on which they are based has been invalidated. If these therapies are promulgated based on the invalidated theory, they are classified as pseudoscientifically oriented psychotherapies (POP). Here’s an example from their article of a commonly utilized approach, neurolinguistic programming, that is based on a disproven theory:
An interesting shift from SOPs to POPs is illustrated by neurolinguistic programming. Once an interesting system (e.g., Category IV of SOPs, according to our classification), it is now seen largely as a POP (Category VII) because although its theory was invalidated by a series of studies (for details, see Heap, 1988; Lilienfeld et al., 2003), it continues to be promoted in practice based on the same theory, as if it were valid (p. 95).
Let’s break this down a little bit. While there is a general lack of evidence for the effectiveness of NLP, there is a greater consensus that the underlying theory contradicts basic research in neuroscience or psychology. NLP uses many scientific sounding but empty terms such as pragmagraphics, surface structure, non-accessing movement, metamodeling, metaprogramming, and submodalities. While these terms form the theoretical foundation for much of the NLP techniques and sound scientific, they have not stood up to scientific scrutiny and thus the term pseudoscientific applies to this therapy.
Science cannot be stagnant. It is ever evolving and needs to be modifiable based on what the data suggest. In order for science to progress and produce effective treatments over time, good theory is needed. Theory is what allows scientists to make sense of the findings that are observed and guides new research. Brute force empiricism, without theory, leads to a lot of blind paths and wasted energy. I’m heartened to see a leading journal discussing alternate schemes for evaluating the scientific credibility of therapies that focus on mechanisms of action, theory, and incorporates understanding derived from basic science.
David, D., & Montgomery, G. H. (2011). The Scientific Status of Psychotherapies: A New Evaluative Framework for Evidence-Based Psychosocial Interventions. Clinical Psychology: Science and Practice. Volume 18, Issue 2, pages 89–99.