Enough already: Let’s move on from meta-analyses of psychoanalytic psychotherapy and do the hard work of quality studies

Enough already: Let’s move on from meta-analyses of psychoanalytic psychotherapy and do the hard work of quality studies

The past few years has seen an increasing push to demonstrate the legitimacy of long-term psychodynamic and psychoanalytic therapies (e.g., Leichsenring & Rabung, 2008). It seems proponents of psychodynamic therapy are trying to play catch up. There’s an enormous amount of research support for cognitive behavioral approaches; by contrast, controlled research for psychodynamic approaches is sparse.

Since long-term psychoanalytic psychotherapy is a mouthful, we’ll follow the cue of those before us and call it LTPP for short.

It’s unfortunate there’s not a lot of controlled research on LTPP, as I think controlled studies carefully examining the processes and outcomes of psychodynamic therapies could only enhance our understanding of treatment. As a way to bolster support, some psychodynamic researchers have taken what studies exist and published meta-analyses of the existing research. Some of the recent ones concluded that LTPP is an effective treatment for a variety of psychiatric conditions (De Maat, 2009; Leichsenring & Rabung, 2008, 2011). These findings were not without controversy, however.

You may have noticed, for example, that Leichsenring and Rabung are listed twice. There’s a reason for this: their 2008 meta-analysis was widely criticized for miscalculating effect sizes. According to critics, the researchers had simply looked at pre-post changes (within-group difference) rather than comparing LTPP against the control conditions (between-group differences). The researchers redid their analysis in the 2011 meta-analyses and found LTPP was effective, albeit to a lesser degree.

Although I think meta-analyses on psychodynamic therapy have been over-played of late, I was excited about a new one published in Clinical Psychology Review (Smit et al., 2012). This article is a collaboration of Dutch researchers and John Ioannidis. I perked up at the mention of Dr. Ioannidis’ involvement.

Who is John Ioannidis?

Not to give short shrift to the others involved, but I was really excited by Ioannidis’ presence on this article. Ioannidis, a medical researcher with several academic appointments, has become one of the foremost experts in the credibility of medical research. He published a hugely influential paper arguing that most medical findings are inaccurate, and he was even profiled in The Atlantic. With his name attached to this piece, I could be confident that the methodology of this meta-analysis had been scrupulously thought out and executed.

A little background on meta-analysis

Meta-analyses are a way of consolidating a number of studies on a particular area of focus that allows for comparison across studies. Researchers may then draw more general conclusions from a bunch of data. This is done by converting the results from each individual study into effect sizes.

Like any tool, meta-analyses are only as good as the way they are used. Researchers make decisions about what studies to include (and not to include), what outcomes to look at, and how to run the analyses. As they say, “garbage in, garbage out.” Moreover, meta-analyses are no substitute for rigorously controlled studies.

The bottom line: meta-analyses of low quality research lead to low quality conclusions

The reason why I’m hoping this article will be the final word on this topic for now is that it ultimately points to the need for more high quality data.

In contrast to previous meta-analyses, the researchers in this study had difficulty drawing firm conclusion about LTPP because the available research was generally of low quality. Their main criticism is that LTPP was often compared against substandard treatments. The authors call these “straw man” comparisons, as there is little reason to believe these control conditions are effective. In the few studies that compare LTPP against evidence-based treatments such as dialectical behavior therapy, LTPP does not fare so well, according the researchers.

What this means is that without well-controlled studies of LTPP against established treatments for specific psychiatric problems. It is difficult to gauge the effectiveness of LTPP. The few highly quality studies available suggest that when LTPP is compared against bona fide treatments, it doesn’t appear to be particularly effective. Hopefully, researchers will now take a break from meta-analysis and focus their efforts towards creating more high quality, controlled studies comparing LTPP to treatments with a strong track record.

Let’s move on and do the work…

I think we’ve seen enough meta-analyses on LTPP for the time being. If a strong argument for LTPP is to be made, it will require a focus on quality, controlled research that compares LTPP to bona fide treatments for specific conditions.

But don’t take my word for it: I highly recommend reading the study yourself. For a scientific article, it’s actually quite lucid and readable. I obtained it by following James Coyne’s suggestion in the blog post that alerted me to this article, and emailing the author, Arnoud Arntz, who quickly and thoughtfully sent me a copy: