The Trojan Horses of Medicine

Dodgy science is being smuggled into medical journals thanks to a loophole in the regulations, say Italian psychiatrists Barbui and Cipriani in an important article.

They focus on agomelatine, a recently-approved antidepressant. But their point applies to all of medicine, not just psychiatry.

Here's the problem. Nowadays, major medical journals have rules governing systematic reviews and meta-analyses of clinical trial data. If you want to review the evidence about how well a certain drug works, or its safety, you've got to do it properly. You have to consider all of the data, not just focus on the results that suit you. And so on.

However, these rules don't apply to "narrative" review papers, which is a broad term meaning any kind of article meant to give a discussion of the pharmacology, history, chemistry etc. behind a particular drug. For a narrative review, there are no rules.

In particular, you can write about the clinical trial data in such articles with no restrictions. Unlike in a proper systematic review, you can cherry-pick trials and so on to your heart's content. Some narrative reviews have so much clinical data in them that they end up being, in effect, a bad systematic review. One that would never have been deemed acceptable as a systematic review.

Barbui and Cipriani argue that narrative reviews are often used in this way, namely to paint drugs in a positive light. In the case of agomelatine, they mention a number of recent narrative reviews which were supposedly about the drug's mechanism of action, but which actually contained extensive (but biased) reviews of the clinical trial data.

It's not hard to see how pharmaceutical companies might take advantage of this process.

However, the problem is surely not limited to agomelatine. It's a loophole that affects every branch of medicine:

Most medical journals require adherence to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA). It is an evidence-based minimum set of items for reporting in systematic reviews and meta-analyses. Adherence to PRISMA is not required in review articles dealing with basic science issues as these articles are not focused on clinical trials.

In practice, however, the agomelatine case indicates that clinical data are regularly included and reviewed with no reference to the rigorous requirements of the PRISMA approach. These articles have this way became a modern Trojan horse for reintroducing the brave old world of narrative-based medicine into medical journals.

How do we stop this? It's simple, the authors say: just make all references to clinical data subject to PRISMA, or other accepted regulations, whatever the supposed 'primary focus' of the paper:

We argue that medical journals should urgently apply this higher standard of reporting, which is already available, easy to implement and inexpensive, to any form of clinical data presentation.

Of course, there are plenty of good narrative reviews that really do cover the pharmacology or other science in a useful way. The problem is not narrative reviews as such, but the way they're used.

Barbui, C., and Cipriani, A. (2012). Agomelatine and the brave old world of narrative-based medicine Evidence-Based Mental Health, 15 (1), 2-3 DOI: 10.1136/ebmh.2011.100485