How I Read Papers

Last year I blogged about how I write blog posts. I don't really have anything to add to that, so here's some advice on how I read scientific papers - both the ones I read for my day job, and the ones I blog about.


Software: If you read papers you need PubCrawler. It's free, and it's the best thing since PubMed, because it automatically searches PubMed for you and emails you the results. Second, you need a reference manager program. I use EndNote, but there are others, including various free ones. They're indispensable.

PubCrawler sends you lists of new papers you might want to read. A reference manager lets you to keep track of what you've read, and what you need to read in future; it lets you make notes on papers (see below), search them etc. and best of all it lets you insert them into Word or whatever and automatically generates a References list. If you're not using these tools, you're making life much harder than it should be.

Deciding What To Read: There are a lot of papers out there. My PubCrawler includes a search term for "antidepressants", which nets about 10 per day; one for "autism", about 5 per day; one for various brain regions I'm interested in, up to 50 per day, another for neurotransmitters I'm into, also 50...

So you need a triage system. I mentally put papers into 3 categories, based purely on the titles:

1. Irrelevant - don't even click on it. I'd say about 80% of PubCrawler hits fall into this category.
   
2. Somewhat interesting - read the abstract. 15%.
   
3. Very interesting - read the whole thing. 5%.

Reading papers: Start with the abstract. Then read the Introduction, as it's usually a pretty good summary of previous work. I'll skip this only if I know all the existing literature (very rare). Then, head to the first paragraph of the Discussion: this typically contains a summary of the main results in non-technical language.

Finally, I'll skim the Methods and the Results. If something seems unusual, dodgy, or especially interesting, I'll go back and read these fully, but most of the time I don't bother. The remainder of the Discussion is generally just speculation, and rarely worth reading.

All that applies to original experimental articles. For review papers, if I read them at all I read them straight through; a well-written review should all be useful. A bad review is no use at all. If you start reading a review, and by the end of the first page you're wondering "But what's the point of all this?", it's probably the latter.

Making notes: This is the key to memory, for me at least. If I just read something, I barely remember it the next day let alone next month. Making notes forces you to actually understand it, and then it sticks. I make notes in EndNote for every paper, and even every abstract, I read. Once you get into the swing of it it's a natural part of reading and doesn't take much time.

Here's my notes on one recent paper:

Abstract. NRG1 --> ErbB4 promotes the formation of glutamatergic --> GABA interneuron synapses via stabilizing the PSD-95 at these synapses, but NOT at other synapses i.e. glut --> glut. Therefore, NRG1 contributes to the development of inhibitory signalling. The authors say this is interesting re: SCZ [but I think it's interesting re: autism as well!]

This makes sense, if you're me. Actually, though, I rarely ever read these notes. The point is to make them. You could scribble them on toilet paper and flush them once you're finished and they'd still do their job of boosting your memory.

Here's an uncensored extract from my notes on a paper I didn't like:

Less "medication resistant" patients did better [well that's AWESOME for a treatment that's meant to be an alternative to meds isn't it, you fuck]. They admit that the actual performance was crap NNT=12, but say it would be better if concomitant meds allowed [....well yeah either that or the effect would DISAPPEAR] and that it is equivalent to what would be expected if you gave a new drug or augmentation to this population [but you DIDN'T did you, you are referring to the literature, which is shit]. There's so many conflicts of interest it's almost tragic.

It deserved it, seriously. My comments are [in brackets], obviously.

Again - when I wrote these, I didn't expect to ever read them. The point is that by writing down my comments, I forced myself to make them coherent, and hence made myself remember them. This is crucial: if you only remember what the paper said, and not the fact that when you read it, you burst out laughing in disbelief, you'll go away thinking that the paper must have been fine.

Premature Brain Diagnosis in Japan?

Nature has a disturbing article from their Asian correspondent David Cyranoski: Thought experiment. It's open access.

In brief: a number of top Japanese psychiatrists have started offering a neuroimaging method called NIRS to their patients as a diagnostic tool. They claim that NIRS shows the neural signatures of different mental illnesses.

The technology was approved by the Japanese authorities in April 2009, and since then it's been used on at least 300 patients, who pay $160 for the privilege. However, it's not clear that it works.

To put it mildly.

*

NIRS is Near Infra-Red Spectroscopy. It measures blood flow and oxygenation in the brain. In this respect, it's much like fMRI, but whereas fMRI uses superconducting magnets and quantum wizardry to achieve this, NIRS simply shines a near-infra-red light into the head, and records the light reflected back

It's a lot cheaper and easier than MRI. However, the images it provides are a lot less detailed, and it can only image the surface of the brain. NIRS has a small but growing number of users in neuroscience research; it's especially popular in Japan, for some reason, but it's also found plenty of users elsewhere.

The clinical use of NIRS in psychiatry was pioneered by one Dr Masato Fukuda, and he's been responsible for most of the trials. So what are these trials?

As far as I can see (correct me if I'm wrong), these are all the trials comparing patients and controls that he's been an author on:

• Matsuo et al (2000) n=9/10 elderly depressed/controls
• Suto et al (2004) n=10/13/16 depressed/schizophrenia/controls
• Kameyama et al (2006) n=17/11/17 bipolar/depression/controls
• Nishimura et al (2007) n=5/33 panic disorder/controls
• Takizawa et al (2008) n=55/70 schizophrenia/controls
• Uehara et al (2007) n=11/11 eating disorder/controls
• Suda et al (2010) n=27/27 eating disorder/controls

There are also a handful of Fukuda's papers in Japanese, which I can't read, but as far as I can tell they're general discussions rather than data papers.

So we have 342 people in all. Actually, a bit less, because some of them were included in more than one study. That's still quite a lot - but there were only 5 panic patients, 30 depressed (including 9 elderly, who may be different), 38 eating disordered and just 17 bipolar in the mix.

And the bipolar people were currently feeling fine, or just a little bit down, at the time of the NIRS. There are quite a lot of other trials from other Japanese groups, but sticking with bipolar disorder as an example, no trials that I could find examined people who were currently ill. The only other two trials, both very small, were in recovered people (1,2).

Given that the whole point of diagnosis is to find out what any given patient has, when they're ill, this matters to every patient. Anyone could be psychotic, or depressed, or eating disordered, or any combination thereof.

Worse yet, in many of these studies the patients were taking medications. In the 2006 depression/bipolar paper, for example, all of the bipolars were on heavy-duty mood stabilizers, mostly lithium; plus a few antipsychotics, and lots of antidepressants. The depressed people were on antidepressants.

There's a deeper problem. Fukuda says that NIRS corresponds with the clinical diagnosis in 80% of cases. Let's assume that's true. Well, if the NIRS agrees with the clinical diagnosis, it doesn't tell us anything we didn't already know. If the NIRS disagrees, who do you trust?

I think you'd have to trust the clinician, because the clinician is the "gold standard" against which the NIRS is compared. Psychiatric diseases are defined clinically. If you had to choose between 80% gold and pure gold, it's not a hard choice.

Now NIRS could, in theory, be better than clinical diagnosis: it could provide more accurate prognosis, and more useful treatment recommendations. That would be cool. But as far as I can see there's absolutely no published evidence on that.

To find out you'd have to compare patients diagnosed with NIRS to patients diagnosed normally - or better, to those randomized to get fake placebo NIRS, like the authors of this trial from last year should have done. To my knowledge, there have been no such tests at all.

*

So what? NIRS is harmless, quick, and $160 is not a lot. Patients like it: “They want some kind of hard evidence,” [Fukuda says], especially when they have to explain absences from work. If it helps people to come to terms with their illness - no mean feat in many cases - what's the problem?

My worry is that it could mean misdiagnosing patients, and therefore mis-treating them. Here's the most disturbing bit of the article:

...when Fukuda calculates his success rates, NIRS results that match the clinical diagnosis are considered a success. If the results don’t match, Fukuda says he will ask the patient and patient’s family “repeatedly” whether they might have missed something — for example, whether a depressed patient whose NIRS examination suggests schizophrenia might have forgotten to mention that he was experiencing hallucinations.

Quite apart from the implication that the 80% success rate might be inflated, this suggests that some dubious clinical decisions might be going on. The first-line treatments for schizophrenia are quite different, and rather less pleasant, than those for depression. A lot of perfectly healthy people report "hallucinations" if you probe hard enough. "Seek, and ye shall find". So be careful what you seek for.

While NIRS is a Japanese speciality, other brain-based diagnostic or "treatment personalization" tools are being tested elsewhere. In the USA, EEG has been proposed by a number of groups. I've been rather critical of these methods, but at least they've done some trials to establish whether this actually improves patient outcomes.

In my view, all of these "diagnostic" or "predictive" tools should be subject to exactly the same tests as treatments are: double blind, randomized, sham-controlled trials.

Cyranoski, D. (2011). Neuroscience: Thought experiment Nature, 469 (7329), 148-149 DOI: 10.1038/469148a

fMRI Scanning Salmon - Seriously.

Back in 2009, a crack team of neuroscientists led by Craig Bennett (blog) famously put a dead fish into an MRI scanner and showed it some pictures.



They found some blobs of activation - when they used an inappropriately lenient statistical method. Their point, of course, was to draw attention to the fact that you really shouldn't use that method for fMRI. You can read the whole paper here. The Atlantic Salmon who heroically volunteered for the study was no more than a prop. In fact, I believe he ended up getting eaten.

But now, a Japanese team have just published a serious paper which actually used fMRI to measure brain activity in some salmon: Olfactory Responses to Natal Stream Water in Sockeye Salmon by BOLD fMRI.

How do you scan a fish? Well, like this:

A total of 6 fish were scanned. The salmon were immobilized by adding an anaesthetic (eugenol) and a muscle relaxant (gallamine) to their tank of water. Then, they were carefully clamped into place to make sure they really wouldn't move, while a stream of oxygenated water was pumped through their tank.

Apart from that, it was pretty much a routine fMRI scan.

Why would you want to scan a fish? This is where the serious science comes in. Salmon are born in rivers but they swim out to live in the ocean once they reach maturity. However, they return to the river to breed. What's amazing is that salmon will return to the same river that they were born in - even if they have to travel thousands of miles to get there.

How they manage this is unclear, but the smell (or maybe taste) of the water from their birth river has long been known to be crucial at least once they've reached the right general area (see here for a good overview). Every river contains a unique mixture of chemicals, both natural and artificial (pollutants). Salmon seem to be attracted to whatever chemicals were present in the water when they were young.

In this study, the fMRI revealed that relative to pure water, home-stream water activated a part of the salmon's telencephalon - the most "advanced" part (in humans, it constitutes the vast majority of the brain; in fish, it's tiny). By contrast, a control scent (the amino acid L-serine) did not activate this area, even though the concentration of L-serine was far higher than that of anything in the home-stream water. How this happens is unclear, but further studies of the identified telencephalon area ought to shed more light on it.

So fishMRI is clearly a fast-developing area of neuroscience. In fact, as this graph shows, it's enjoying exponential growth and, if current trends continue, could become almost as popular as scanning people...

Link: Also blogged at NeuroDojo.

Bandoh H, Kida I, & Ueda H (2011). Olfactory Responses to Natal Stream Water in Sockeye Salmon by BOLD fMRI. PloS one, 6 (1) PMID: 21264223

"Packing" Autistic Kids: A French Scandal

Back in the bad old days of autism they thought it was caused by "refrigerator mothers".


Well, right now, some psychiatrists have decided that the best treatment for autism is something not that far removed from sticking them in a refrigerator - literally. Enter "Le Packing", which is the target of an unprecedented consensus statement just out from a list of 18 big-name autism experts (available free here).

This alleged therapy consists of wrapping the patient (wearing only underclothes or naked in the case of young children) several times a week during weeks or months in towels soaked in cold water (10°C to 15°C). The individual is wrapped with blankets to help the body warm up in a process lasting 45 minutes, during which time the child or adolescent is accompanied by two to four staff persons.

The alleged goal of this technique is to “allow the child to rid him- or herself progressively of its pathological defense mechanisms against archaic anxieties,” by achieving “a greater perception and integration of the body, and a growing sense of containment.”

No, really. Frankly, they could have stopped there, because the description is condemnation enough, but they go on to write:

We have reached the consensus that practitioners and families around the world should consider this approach unethical. Furthermore, this “therapy” ignores current knowledge about autism spectrum disorders; goes against evidence-based practice...and, in our view, poses a risk of preventing these children and adolescents from accessing their basic human rights to health and education.

Le Packing, as the name suggests, originated in France, and its use seems to be confined to France and other French-speaking areas. This is the first I'd ever heard of it. Little has been written about it in English (though see this long article and this piece from 2007) so here's my loose translation of the the article on the French Wikipedia:

• Packing is used in children with autism, but also in others: psychotic adults (specifically when they're recovering from an acute psychotic episode), in the elderly, etc.
• It's intended to restore "awareness of the body image".
• It's extremely controversial. Well, duh.
• The technique was invented, in France, by a "controversial American psychiatrist" called M. A. Woodburry. It was intended for the treatment of severely autistic children and adolescents, especially those with severe behavioural problems such as self-harm, aggression, and refusal to eat.
• The patient is wrapped in towels covered in cold water: two towels for the torso, and one for each arm and leg. They're then additionally wrapped in a sheet and then blankets, over the towels. The cold water quickly warms up thanks to body heat: the child is never actually hypothermic.
• After this session, the child is "frictionné" (I guess this means massaged) and taken to their living quarters and offered a snack "in a friendly atmosphere".
• Le Packing is intended to recover a physical sense of their own body. It should be used as part of a wider package of care, and only with the consent of the patient's parents.
• The cold water is optional; some, e.g. a Dr A. Gillis, use warm water nowadays. The key point is the restraint, i.e. the fact that their attempts to move their body are restricted temporarily. Hence "le packing", huh.
  
• The scientific status of Le Packing is controversial. A group called "Léa pour Samy" say it should be banned, and replaced by the (much more orthodox) method of ABA. However, in 2007, authorities approved a randomized controlled trial led by a "Dr Goeb" of the CHU hospital in Lille.
• Critics accuse Le Packing of being an unethical, inhuman and degrading treatment, maybe even torture. There are allegations of cases in which the towels were much colder than 10°C, e.g. straight out of the freezer.
• There are also allegations of its use without parental consent. A Professor Pierre Delion, of the CHU in Lille, reportedly defended this in remarks to The Lancet "if a child is in danger following a road accident, you do not wait for the parents' agreement to give him a transfusion." But this is actually a misquote. In the Lancet piece, he was referring to the patient's consent and said parental consent was always sought.
  
• In 2009 a government minister told the French Senate that Le Packing should only be used under strictly controlled conditions according to a protocol - but others, e.g. the "Léa pour Samy" group, want it banned altogether.
  

This rather speaks for itself, but I'll say this. If someone is suffering these kinds of severe behavioural disturbances, the temptation to do something dramatic must be intense. Indeed, if someone's disturbed to the point of trying to mutilate themselves, or refusing to eat, almost by definition you're going to have to restrain them, either physically or with sedatives, temporarily. While Le Packing may be a French peculiarity, it's not like psychiatrists in other countries never resort to drastic measures.

Amaral D, Rogers SJ, Baron-Cohen S, Bourgeron T, Caffo E, Fombonne E, Fuentes J, Howlin P, Rutter M, Klin A, Volkmar F, Lord C, Minshew N, Nardocci F, Rizzolatti G, Russo S, Scifo R, & van der Gaag RJ (2011). Against le packing: a consensus statement. Journal of the American Academy of Child and Adolescent Psychiatry, 50 (2), 191-2 PMID: 21241956

When "Healthy Brains" Aren't

There's a lot of talk, much of it rather speculative, about "neuroethics" nowadays.

But there's one all too real ethical dilemma, a direct consequence of modern neuroscience, that gets very little attention. This is the problem of incidental findings on MRI scans.

An "incidental finding" is when you scan someone's brain for research purposes, and, unexpectedly, notice that something looks wrong with it. This is surprisingly common: estimates range from 2–8% of the general population. It will happen to you if you regularly use MRI or fMRI for research purposes, and when it does, it's a shock. Especially when the brain in question belongs to someone you know. Friends, family and colleagues are often the first to be recruited for MRI studies.

This is why it's vital to have a system in place for dealing with incidental findings. Any responsible MRI scanning centre will have one, and as a researcher you ought to be familiar with it. But what system is best?

Broadly speaking there are two extreme positions:

1. Research scans are not designed for diagnosis, and 99% of MRI researchers are not qualified to make a diagnosis. What looks "abnormal" to Joe Neuroscientist BSc or even Dr Bob Psychiatrist is rarely a sign of illness, and likewise they can easily miss real diseases. So, we should ignore incidental findings, pretend the scan never happened, because for all clinical purposes, it didn't.
2. You have to do whatever you can with an incidental finding. You have the scans, like it or not, and if you ignore them, you're putting lives at risk. No, they're not clinical scans, they can still detect many diseases. So all scans should be examined by a qualified neuroradiologist, and any abnormalities which are possibly pathological should be followed-up.

Neither of these extremes is very satisfactory. Ignoring incidental findings sounds nice and easy, until you actually have to do it, especially if it's your girlfriend's brain. On the other hand, to get every single scan properly checked by a neuroradiologist would be expensive and time-consuming. Also, it would effectively turn your study into a disease screening program - yet we know that screening programs can cause more harm than good, so this is not necessarily a good idea.

Most places adopt a middle-of-the-road approach. Scans aren't routinely checked by an expert, but if a researcher spots something weird, they can refer the scan to a qualified clinician to follow up. Almost always, there's no underlying disease. Even large, OMG-he-has-a-golf-ball-in-his-brain findings can be benign. But not always.

This is fine but it doesn't always work smoothly. The details are everything. Who's the go-to expert for your study, and what are their professional obligations? Are they checking your scan "in a personal capacity", or is this a formal clinical referral? What's their e-mail address? What format should you send the file in? If they're on holiday, who's the backup? At what point should you inform the volunteer about what's happening?

Like fire escapes, these things are incredibly boring, until the day when they're suddenly not.

A new paper from the University of California Irvine describes a computerized system that made it easy for researchers to refer scans to a neuroradiologist. A secure website was set up and publicized in University neuroscience community.

Suspect scans could be uploaded, in one of two common formats. They were then anonymized and automatically forwarded to the Department of Radiology for an expert opinion. Email notifications kept everyone up to date with the progress of each scan.

This seems like a very good idea, partially because of the technical advantages, but also because of the "placebo effect" - the fact that there's an electronic system in place sends the message: we're serious about this, please use this system.

Out about 5,000 research scans over 5 years, there were 27 referrals. Most were deemed benign... except one which turned out to be potentially very serious - suspected hydrocephalus, increased fluid pressure in the brain, which prompted an urgent referral to hospital for further tests.

There's no ideal solution to the problem of incidental findings, because by their very nature, research scans are kind of clinical and kind of not. But this system seems as good as any.

Cramer SC, Wu J, Hanson JA, Nouri S, Karnani D, Chuang TM, & Le V (2011). A system for addressing incidental findings in neuroimaging research. NeuroImage PMID: 21224007