Friday, 30 December 2011

How Realistic is fMRI?

How representative are fMRI experiments? Is "the brain" that we investigate with fMRI the same brain that we use outside the MRI scanner?

A new paper from Bernhard Hommel and colleagues of Leiden in the Netherlands offers some important caveats. They looked to see what effect playing some recorded MRI scanner sounds had on people's ability to perform some simple cognitive tasks, while sitting outside the scanner.

MRI is notoriously noisy. When you have an MRI scan you have to wear earplugs to protect against the sound but they only block out some of it. Opinions differ on whether the sound is pleasant or not. Personally I find the repetitive tick-tock rather soothing now, but then I've heard it many times over the years. First-timers can find it quite overwhelming.

Anyway, Hommel et al found that while scanner noise had no overall effects on reaction time or accuracy, it actually improved performance on three measures of "cognitive control".

For instance in a task in which participants had to respond to the colour of a circle by pressing the left or the right arrow key, they were slower to react when the circle appeared on the "wrong" side of the screen, i.e. on the left when the correct answer was the right arrow. This slowing of responses caused by a stimulus-response clash is called the Simon effect.

The results showed that the Simon effect was reduced by noise. The same thing happened in two other studies: noise meant better performance.

All of the noise effects were modest and the sample sizes were also quite small (14-18 per task, with everyone studied twice, noisy vs silent) but this paper joins a number of others raising questions about the representativeness of fMRI, with evidence that fMRI activates the brain and maybe even improves mood (although I doubt that last one).

The authors' interpretation is that the noise made people pay more attention to the tasks, to compensate for the distraction, and that this means that fMRI studies may be biased in their measurements of cognitive control:
Generalizing from fMRI findings to behavioral observations and vice versa seems to be more problematic than commonly thought, at least as far as control  processes are concerned. In a sense, then, investigating cognitive processes by means of  fMRI... is inevitably facing Heisenberg’s (1927) uncertainty principle, according to which the act of measurement can change what is being measured.
To my mind the biggest weakness of this is that it only looked at noise. While scanners are noisy, that's not the only distracting thing about them: during an fMRI study you also have to lie down, in a small confined tube, and your only way to see the "screen" on which experimental stimuli are shown is indirectly via a small mirror which often doesn't give a good view.

So ironically, I'm not sure how realistic this study is...


ResearchBlogging.orgHommel, B., Fischer, R., Colzato, L., van den Wildenberg, W. and Cellini, C. (2011). The effect of fMRI (noise) on cognitive control. Journal of Experimental Psychology: Human Perception and Performance DOI: 10.1037/a0026353

21 comments:

practiCal fMRI said...

Yup! It's realistic for brain activity when lying down (possibly sleepily) in a noisy environment... For tennis, or driving a car? Not so much. Still, one hopes that my brain doesn't change fundamentally when I lay down. Or play tennis, come to that.

Neuromancy said...

I guess it was always one of the standard tropes of experiment evaluation/discussion at school that "experiments aren't necessarily ecologically valid". Nice to see people taking a stab at seeing how fMRI experiments differ from standard cog psych.

@practiCal_fMRI I have a vague memory of a paper about the effect of lying down on blood flow, and how it might settle in the back of the head. I don't think there was a functional effect, but presumably it would modulate the BOLD signal?

Ivana Fulli MD said...

Neurotypical subjects might react by concentrating on one task when suffering a noise but schizophrenic diagnosed persons have been described as unable to selectively shut off unwanted auditory information.

It might well be that so called schizophrenics are just unable to shut off their auditory part of the temporal lobe when they are speaking loud or to themselves in their head like anybody else. fRMI people have published on that.

It would make it uneasy for them to know that nobody else than themselves is speaking in their head.

A work from 1981 –pay wall for me and I could reread only the abstract today

- might be of interest to you fMRI researchers focusing on that post:

I cite the abstract:
“(...)One condition required the active monitoring of material presented and resulted in the greatest decrease in hallucinatory experiences. (…)it is not stimulation per se, but rather the extent to which structure is present and attended to, that is important in determining the occurrence of hallucinatory phenomena.”

http://bjp.rcpsych.org/content/139/2/122.abstract

Also, autistic aspies are notorious for being often overwhelmed by sensory stimulations and hating new situations. By the way a pro bono client of mine telephoned me a few days ago to ask for my help with a one month pre-contract job he has to start next Monday and incidentally told me that he participated in another fMRI study in Lyon. I didn’t dwell on that but it must be the fourth or fifth time for that client. I do not think his brain will react the same since he knows very well the place now and is used to travel there. Anyway, after the first time, my client complained about the noise of the machinery and having suffered a lot from it to the point of asking medication for a headache afterward.

PS: When psychiatrists were being simplistic and overoptimistic in social psychiatry or in drugs following the lead of Dr Stahl- clever non psychoanalyst psychologists were doing serious work (a couple of psychiatrists from Holland were present too).
There are psychological theories and psychological thinking as wrote angrily an anonymous in your “confused” very brave and humane post… Since I am strongly against the psychologists right to prescribe drugs with serious side-effects, I am always pleased to show off my knowledge of all their good works and very interesting body of psychological thinking.

Happy new year neuroskeptic

practiCal fMRI said...

@Neuromancy Blood pressure definitely changes, so there may well be a systematic change in the global vascular dynamics, that's true. But the brain's pretty darn insistent on homeostasis. I don't know whether there would be significant effects on neurovascular coupling just from lying down. It's something I want to measure.

We've been looking into MRI-compatible kit that can do time series "in magnet" BP measures without a cuff, to assess the possible differential BP in response to certain tasks, i.e. between conditions. Heart rate and respiration rate changes with task (e.g. with arousal) are a concern for similar reasons.

jamzo said...

from wikipedia - magnetic resonance imaging entgry

"Economics of MRI

MRI equipment is expensive. 1.5 tesla scanners often cost between US$1 million and US$1.5 million. 3.0 tesla scanners often cost between US$2 million and US$2.3 million. Construction of MRI suites can cost up to US$500,000, or more, depending on project scope.

MRI scanners have been significant sources of revenue for healthcare providers in the US. This is because of favorable reimbursement rates from insurers and federal government programs. Insurance reimbursement is provided in two components, an equipment charge for the actual performance of the MRI scan and professional charge for the radiologist's review of the images and/or data. In the US Northeast, an equipment charge might be $3,500 and a professional charge might be $350 [48] although the actual fees received by the equipment owner and interpreting physician are often significantly less and depend on the rates negotiated with insurance companies or determined by governmental action as in the Medicare Fee Schedule. For example, an orthopedic surgery group in Illinois billed a charge of $1,116 for a knee MRI in 2007 but the Medicare reimbursement in 2007 was only $470.91.[49] Many insurance companies require preapproval of an MRI procedure as a condition for coverage.

In the US, the Deficit Reduction Act of 2005 significantly reduced reimbursement rates paid by federal insurance programs for the equipment component of many scans, shifting the economic landscape. Many private insurers have followed suit.[citation needed] In France, the cost of an MRI exam is approximately 150 Euros. This covers three basic scans including one with an intravenous contrast agent, as well as a consultation with the technician and a written report to the patient's physician.[citation needed]"

Ivana Fulli MD said...

PractiCal FRMI,

Derman and Wilmott wrote a 2009 manifesto urging to modesty with a parody of the Hippocratic Oath

http://www.pdfdownload.org/pdf2html/view_online.php?url=http%3A%2F%2Fwww.ederman.com%2Fnew%2Fdocs%2Ffmm.pdf) :


One of my sons had a good schooling in math and is doing a PhD in finance and it is why I became interested in that manifesto

Do you not think it applies to fRMI threatening looking researchers like yourself with little modifications like a psychologist blogger proposed to adapt it to psychology research ?

http://fixingpsychology.blogspot.com/2011/11/modelers-hippocratic-oath.html

The Derman and Wilmot Modeler ‘s Hyppocratic Oath
comes at the end of the manifesto just after

MODELERS OF ALL MARKETS, UNITE! You have nothing to lose but your illusion.

It goes like that :

• I will remember that I didn’t make the world, and it doesn’t satisfy my equations.
• Though I will use models boldly to estimate value, I will not be overly impressed by mathematics.
• I will never sacrifice reality for elegance without explaining why I have done so.
• Nor will I give the people who use my model false comfort about its accuracy. Instead, I will make explicit its assumptions and oversights.
• I understand that my work may have enormous effects on society and the economy, many of them beyond my comprehension.



PS: One of my sons had a good schooling in math and is doing a PhD in finance and it is why I became interested in that manifesto.

But it tooks me reading a very smart psychologist blogger proposition to adapt it to psychology research, to realize that it might applies to many researchers hubris due to the fact that they can read equations like other can read music-without being Mozart or Britten.

Anonymous said...

Ivana, exactly. fMRI is a model. Not a representation of reality, but an interpretation of reality.

Way to much stock is put into the accuracy of those models. And even worse far reaching conclusions are drawn from it which are more likely mere wishful thinking then actual results.

What's needed is a placebo fMRI, to see how much personal bias influences results.

And even so, knowing that oxygen reduction takes place is not an indicator that the eventual place where it happens is doing anything related to the event you instigate. It might as well be a suppressing action which hides the real activating action.

In other words: nice toy that makes nice colored pictures but as useful as a crystal ball.

Ivana Fulli MD said...

petrossa,

I am sorry if what I wrote do not reflect clearly my thinking.

To my mind fMRI is a wonferful tool as a laboratory test for clinicians and as a way toward progress in knowledge.

What I object to is premature or dishonest claims about the results of research.

On ethical grounds, I would like to be sure that research is done properly in order not to let down the aspies who put a lot of efforts when the agree to be the subject of research.

PractiCal fMRI seems to me to have a restricted view about that point, although he is doing a very good job in asking for technical state of the art expertise.

It is a pity to my mind that the clinician knowledge and the "psychological thinking" in other world the psychologists mainly body of research on psychological symptoms -or more precisely may be psychological complaints - are often ignored by the psychiatrists and the neuroscientist.

I am very hopeful that if they organize properly the fMRI researchers might help the world understand much more about hallucinations for example.

And I am not even against math oriented persons doing PhD in finance as long as they are socially responsable.

practiCal fMRI said...

Not sure where this thread is going or why it may have changed direction, but for what it's worth, I am agnostic on fMRI as a method. I'm a chemist/physicist with a penchant for experimentation. What fMRI is or isn't used for, what it does or doesn't mean, is not for me to say. My role, such as I have one, is to try to ensure that whatever IS done with an fMRI experiment is done as well as it can be done experimentally. In doing experiments one wants to take into account as many factors as possible, tending towards the ideal number of experimental variables: one. (In the case of fMRI: neural activity differences.)

Now, what I would say about fMRI results (as opposed to the conduct of the experimental method) is this: when the experimenter has controlled for as many extraneous factors as possible, and when the experimental design has been properly balanced, and when the statistical models have been properly applied, what we are left with is no more (or less) valid than any other correlation in science. What it means is another question entirely! Because, as we all know, correlation does not imply causation!!

I'm relying on people like Neuroskeptic and the rest of the neuroscience community to actually interpret this stuff. It's just blobs to me, too!

Ivana Fulli MD said...

practiCal fRMI,

So no offense taken in my stating that you have a limited point of you.

I never fish for compliment, I hope, but for answers, often.

Thanks for answering and happy new year.

Neuroskeptic said...

practiCal fMRI: "Not sure where this thread is going or why it may have changed direction"

Neither am I ;) In future can people try to keep comment threads on topic.

"I'm relying on people like Neuroskeptic and the rest of the neuroscience community to actually interpret this stuff. It's just blobs to me, too!"

In my more pessimistic moments I worry that fMRI will never reach its full potential because in order to make proper use of it, one would have to be a fully paid-up physicist & expert in MRI methodology and a leading neuroscientist able to ask sensible questions that put those techniques to use.

I don't think the situation is as bad as that, really, because effective collaboration can work wonders. But you see the point...

Ivana Fulli MD said...

Neuroskeptic,

The neuroscientists also need to know about the clients they experiment on when they use autistic or schizophrenic persons.

Last year, at the Nat Autistic Soc professional Conference in Manchester , aspies were in the audience and some were speakers.A good looking, smart, energetic neuroscientist professor did during his lecture and questionning things therapists are supposed to help aspies get rid of: he called a lady "orange hair" to give her the priviledge of asking him a question (we teach that no matter how striking is another person body, you never ever designate her person with the fat lady, the short man,the orange hair etc...) and the same orator made a lewd implicit joke about what he does with his wife but only after a few beers (when aspie are not very good with non-explicit material).

It is 3 years at least that I listen to his research on autism and the man do not know much about autistic persons...

Who said that in sciences fortune smiles to the prepared minds?

PS: If a mecchanical noise is very disturbing for autistics and make auditory hallucinations worst may be some German engineers should work on a new model of earplugs (the custom made earplugs cost a fortune)or find a way to make the machinery less noisy!

Doriannevergreys said...

for all that it is worth... we are talking about the ecological validity of a proxy measure of another proxy measure of another proxy measure (even if bold is a good proxy marker of neuronal activity, and assuming that neuronal activity is a proxy marker of mood!!!)... concluding that being in scanner actually improves the mood is a bit far fetched isn't it... I may be wrong...

Vince said...

That is not what the uncertainty principle says. Nor does putting (1927) next to it change that...

Neuroskeptic said...

Vince: Heh, right. Then again that's what always happens.

I propose Neuroskeptic's Certainty Principle : "Whenever anyone likens something to Heisenberg's Uncertainty Principle, it is certain they'll be wrong"

Anonymous said...

DorianneVeryGrey hit the nail on the head imo. An assumption being assumed to be not an assumption, and therefor assumed to be true.

The nice little dots on the screen could relate to what happens in that brain as much as they couldn't.

Making most of not the largest part of research a waste of resources. Not the least the one where a female researcher published a paper claiming that an orgasm led to higher consciousness when she masturbated whilst being in a scanner. Stupid me i lost the link.

Granted that is an extreme, but 'studies' claiming they have found out that Religious belief passes through the logic center and hence belief is not belief but a conscious logically arrived conclusion.

The good thing about being a young retired interested party is that one can read all day if one wants, most scientists also have day job.

I guess in the last 3 decades i've read more then the average scientist.

My take: Most of it is contradictory.

ivana Fulli MD said...

petrossa,

My guess would be that what is the rarest thing is bright ideas to be tested for the neuroscientists but those who looked about shutting off auditory part of the temporal brain by non auditory hallucinating people / hallucinating ones when they speak (to themselves or to other) have gotten something great.

And for us readers of their publications, the most difficult thing is to trust the neuroscientists and geneticians who can compete with any politician about lies on their achievments: some seem egomaniac despising their audiences to my mind.

They might not be working under the commercial pressure of Big Pharma, but the ego pressure plus foundings research motivation has given the world -and the BBc- pathetic talks from neuroscientists.

When they have a good interesting ideas to be tested, the fMRI research people should unite and think even harder about bias to produce better results,

and meditate the Delmot and Willmer 2009 manifesto and adapt their modeller's vow when they write and give talks about their datas.

NB: Problem is that practiCal fMRI and many a neuroscientist will be happy to consider good science and good use of ressources to put women masturbating in a noisy machine ( without a partner and sex toy).

Also, they do not care those fMRI and chums about the fact that fRMI might act on the delicate balance of the brain (NS has a post on it) and more chokingly that using the same brave adult aspie so many times is a bias source.

Always interesting to read you.

Anonymous said...

I used to be a systems analyst. A good one. My analytic mind spots holes in the fMRI plot large enough to drive truck through.

First of all the basic idea.Reduction in deoxyhemoglobin. Sure it tells something.
Something is going on. What? Well that's wholly interpretative. No way of knowing if there indeed is increased activity pertaining to task at hand other then in very obvious an clearly defined area's.

For the rest it's anybody's guess. As clearly shown by the multitude of studies 'proving' anything the researcher wants to prove.

But if that was all that there is wrong with one could work around that.

But the nicely colored dots on the screen aren't nicely colored ots on the screen but the representation of multivariant statistial analysis of parametrized data.

A computer model of the brain to all intents and purposes. (And anyone who has followed climatology developing knows what that means.)

Data which is in principle just chaotic noise with a slight signal in it.

Doesn't take much to throw that off track. Person is anxious, has naturally occuring irregular bloodflow, hormone attack, anomalous wired brain such undiagnosed PD's, ASD, Schizophrenia etc.

Most of the fMRI studies consist of tiny samples. And it's no good saying the statistically they'll represent the population because that just isn't so.

Recent dutch studies showed a vast overrepresentation of Aspergers in a dutch town housing the Technical University.

Other such conglomerations must exist. To get a truly representative sample you'd need to do a large cohort study with samples drawn across all cultures.

Just as diagnostic criteria manuals are shockingly ethnocentric, so are fMRI studies.

Ivana Fulli MD said...

petrossa,

Thanks for your comment.

I understand that the images are covering a hudge territory on the neuronal scale and I agree of course with the methodology traps.

Still, if the FRMI people really had got it right when they showed that the auditor temporal cortex of auditory hallucinating persons is not shutting oput when they speak themselves, they found something great to know and think about

Anonymous said...

Have we learned anything new about brain function from fMRI? Whenever I have asked this question of neuroscientists it has been met with derision and insult. A simple list would have sufficed,

Besides the making of some glorious careers in neuroscience departments worldwide I see little to associate science with fMRI. Maybe there is something to be learned about brain function from fMRI but given the sloppiness of fMRI studies and the lack of concern about what "accepted" means of processing fMRI data actually does to the data I have little hope that the scientific yield of fMRI will ever justify its cost.

The problem, as I see it, is that the field is contaminated by actors. There are just too many people in the field whose primary concern is their career-as-status. Meanwhile the careers of folks like myself, that have low tolerance for wishful thinking and keep our doubtful mouths shut even amongst our closest colleagues, suffer almost fatal blows to our careers because we can not bring ourselves to openly peddle career-making snake-oil.

The deliberateness of good science is sorely missing in neuroscience. Is this something new? No. Is it endemic in neuroscience alone? No. From science to finance the peddling of snake-oil seems to be tolerated if not wholeheartedly promoted in our present corporate-controlled world.

Ivana Fulli MD said...

Anonymous 5 January 2012 09:23

Actually, I will also blame the BBC and hollywood as well as lazy journalists and put my hope in people like you and the clients.

I suffer everytime I watch on the BBC channel a short news report on a breaking news from fRMI in autism.

With a little piece of fRMI research a London team told the world that they were making important steps toward finding a fRMI lab test for autism and prescribing more wisely!

As I already wrote on that blog, the boss showed at the NAS 2011 professionnal conference that he was just incompetent in autism: he called a woman orange hair and made a lewd joke during his lecture about what he does with his wife but only aftrer a few beers during his talk when aspies and high functionning autists were in the audience and some were speakers.

Some other speakers repeated that you have to teach the aspies not to mention striking physical features or intimate things in public with colleagues or people you do not know!
Plus some aspies are very prudish.


NB: and I am one who only did two very very little pieces of research in my long life.

But,

when I wanted to become a neurologist, a professor I admired was concerned about the dead brains of interesting clients-deseperatly hoping that they will not die during a foreign holidays or refuse to give their brain to science.
We had a time when only the autopsy will give us answers.NB: I am not saying that brain banks are not useful, just that I have hopes for fRMI.