Wednesday, 31 October 2012

The Changing Face of British Suicide

Which jobs are at the highest risk of suicide?

In a fascinating new study, British researchers Roberts, Jaremin and Lloyd show dramatic changes over time. 30 years ago, the worst occupations for suicide were the medical professions. Now, it's blue-collar workers, with coal miners topping the list.

They used official records of UK suicides, comparing 1979-1983 and 2000-2005. Here's the key data (their graphs, my colours)


In the 80s, veterinarians were the most suicidal of all jobs; by 2005, they'd dropped off the Top 30 list entirely. Other healthcare professions, like pharmacists and dentists, likewise disappeared after being high on the rankings.

On the other hand, the most dramatic rise in suicide was among coal miners. They went from #29 to #1, and their rates rose about fourfold. This is not that surprising considering what happened to British coal mining during the 80s...

In general the Top 30 in the 1980s had plenty of white collar workers like engineers, chemical scientists and photographers.

Twenty years later, all of the collars had turned blue. Coal miners were joined by labourers, builders, gardeners, butchers, and others. Strikingly there are no 'professionals' on the modern list, with the interesting exception of musicians and artists (a reflection of the mental illness-creativity link?)

What's more, the correlation between socioeconomic status and suicide rates increased sharply over time. Suicide is now much more of a class issue than it was in the past.

The only constant in this sad picture was the sea: merchant sailors had the #2 spot in 1980, and they kept it in 2005, with the rates almost unchanged. However, this should be taken with a pinch of salt, because the way British suicides at sea are recorded is a bit unusual.

ResearchBlogging.orgRoberts SE, Jaremin B, and Lloyd K (2012). High-risk occupations for suicide. Psychological medicine, 1-10 PMID: 23098158

Tuesday, 30 October 2012

Men and Women: From Earth, Not Mars & Venus?

Another day, another debate over how different men and women are, psychologically speaking. Bobbi Carothers and Harry Reis argue that Men and Women Are From Earth.

Their approach is rather interesting.
We sought to empirically determine whether standard gender differences are better conceived as taxonic or dimensional. Although men and women may differ on average in myriad ways, these differences may be dimensional, reflecting different amounts of a given attribute assessed along a single dimension, or qualitative, sorted into fundamentally distinct categories... this difference has considerable importance for understanding the fundamental nature of gender differences.
Using lots of previously published data (13,000 people) and subjecting it to three different methods of statistical "taxometric analysis", they claim that on most psychological measures, there's no evidence that the two sexes are qualitatively different. This is on things like sexual attitudes, personality, and interest in science according to self-report questionnaires.

They give the following hypothetical example to illustrate the idea (my picture based on theirs)
Men are taller than women and also have shorter hair. If you plot a scatterplot of height vs hair length including both genders, you find a negative correlation. However, there is no such correlation within each gender. So gender is a taxon - in this case. There is something qualitatively different between men and women here.

Their argument is that psychological differences between the genders are, in most cases, not because "male" and "female" are two distinct taxons.

So what? I previously covered a paper called The Distance Between Mars and Venus claiming that the difference between men and women on average are larger than previously thought, if you look at all the differences taken together. That's actually consistent with what Carothers and Reis are saying, I think, because it assumes that each of the differences is dimensional and quantitative.

In other words, maybe sexes differ only by a matter of degree, albeit by a larger degree than you'd think at first glance.

All of this leaves open the question of why they differ on average, though. According to yet another study just out, the size of the gap is correlated with the amount of gender inequality in different countries. Women from places where they have much lower incomes, career prospects, etc. compared to men, also endorse more 'feminine' traits.

Personally I consider the question of gender differences largely open because I'm skeptical of self-report questionnaire measures in psychology; objective measures of actual behaviour, stuff like crime statistics, seems to me more interesting.

The fact that the great majority of sex offenders are male, for example, must mean something; I'm not sure what, but I don't think questionnaires will help us find out...

ResearchBlogging.orgCarothers, B., and Reis, H. (2012). Men and Women Are From Earth: Examining the Latent Structure of Gender. Journal of Personality and Social Psychology DOI: 10.1037/a0030437

Saturday, 27 October 2012

Is fMRI About To Get Fifty Times Faster?

According to a paper just published, a new technique of functional MRI scanning (fMRI) could soon allow neuroscientists to measure brain activity far faster: Generalized iNverse imaging (GIN): Ultrafast fMRI with physiological noise correction

Authors Boyacioglu and Barth claim remarkable things for the technique:
We find that the spatial localization of activation for GIN is comparable to an EPI protocol and that maximum z-scores increase significantly... with a high temporal resolution of 50 milliseconds.
EPI, the current standard fMRI sequence, would have a temporal resolution of 2000 or 3000 milliseconds, so it's about 50 times faster.

Other super-fast fMRI methods already exist (e.g. this one I blogged about), but they've generally achieved speed only at a cost: they've had to either sacrifice spatial resolution to achieve that, or limited themselves to scanning only a small fraction of the brain, or have been more subject to random noise and hence less sensitive.

GIN, however, is said to cover the whole brain, with decent spatial resolution and signal-to-noise ratio. The data can be analyzed in exactly the same way as any other kind. So that's up to fifty times faster with no real drawbacks.

That would be truly revolutionary - as the major limitation of fMRI at the moment is that it's much slower than other methods of recording brain activity.

Check it out: this shows brain activation in response to simple visual stimuli, imaged with bog-standard EPI and GIN:


So this is a big deal... if it does work, I'm sure neuroscientists the world over will be lining up to buy Boyacioglu and Barth a GIN and tonic.

How does it work, and is it all it's cracked up to be? Well, I can't really say: the math is beyond me.

In essence, rather than scanning the brain in 3D, slice by slice (like this), GIN only scans one 2D slice, but then manages to reconstruct the rest of the brain in 3D from just that slice, using dark, forbidden magicks... I mean mathematics. The principle is called parallel imaging and it's been around for several years, but with image quality limitations that GIN claims to have overcome.

Perhaps my more technically-inclined readers will have more insightful comments.

ResearchBlogging.orgBoyacioglu R, and Barth M (2012). Generalized iNverse imaging (GIN): Ultrafast fMRI with physiological noise correction. Magnetic Resonance in Medicine PMID: 23097342

Thursday, 25 October 2012

Gene-Guided Antidepressants?

Over the past couple of years, "Big Pharma" has largely moved away from psychiatric drug development. This shift has been widely discussed.

But another trend has been happening over the same time period - or so it seems to me. This is the rise of small companies who offer techniques for diagnosing mental illness, or predicting which drugs will work best. Generally (it seems) partnerships between venture capitalists and psychiatry (ex-)researchers, these enterprises might be dubbed "Little Pharma".


The latest is a company called AssureRx Health, Inc. According to a paper just published, they offer
a pharmacogenomic algorithm designed to improve the safety and efficacy of prescribing antidepressant and antipsychotic medication... based on the genotyping of both copies of five genes.
From this, you end up with a report giving each drug a rating of green, yellow, or red (see above).

The price is not provided on their website.

According to the paper, they gave 26 depressed patients normal treatment at the discretion of their psychiatrist, while 25 got treatment guided by the AssureRx algorithm. It was non-randomized, and unblinded so there's a clear possibility of a placebo effect.

Anyway, the results were...

For the first 4 weeks of treatment, there was no difference between the two groups whatsoever in terms of depression symptom scores - they both improved. But then by week 8, the unguided patients abruptly got worse, while the AssureRx-guided ones continued to benefit. This is an unusual pattern of improvement in an antidepressant trial.

Previously, I wrote about another Little Pharma antidepressant prediction scheme. It used a  different approach, measuring brain electrical activity using a technique called "rEEG", rather than genetics. But the basic idea is the same... and so are the problems.

As I said last time:
There were two groups and they got entirely different sets of drugs. One group also got rEEG-based treatment personalization. That group did better, but that might have nothing to do with the rEEG: they might have done equally well if they'd just been assigned to [those drugs] by flipping a coin. We cannot tell, from these data, whether rEEG offered any benefits at all.
In the AssureRx paper, we can't even tell whether the two groups got different kinds of drugs, because the meds used aren't reported, but if they did differ then that would offer an alternative explanation for the differences in outcome: maybe the 'guidance' just recommended better drugs overall, with the genes being just a sideshow. Or maybe it's a placebo, as I said.

Moving on, I also wrote... 
What's curious is that it would have been very simple to avoid this issue. Just give everyone rEEG, but shuffle the assignments in the control group, so that everyone was guided by someone else's EEG. So you'd give control Patient 2 the drugs that Patient 1 should have got, and vice versa; swap 3 and 4, 5 and 6, etc.

This would be a genuinely controlled test of the personalization, because both groups would get the same kinds of drugs... and it would allow the trial to be double-blind: in this study the investigators knew which group people were in, because it was obvious from the drug choice...
It is odd that Little Pharma never seem to do such real vs. muddled prediction studies, as they'd be really informative as to whether their approach is a helpful innovation as opposed to an expensive, meaningless red herring. Hmm.

ResearchBlogging.orgHall-Flavin, D., Winner, J., Allen, J., Jordan, J., Nesheim, R., Snyder, K., Drews, M., Eisterhold, L., Biernacka, J., and Mrazek, D. (2012). Using a pharmacogenomic algorithm to guide the treatment of depression Translational Psychiatry, 2 (10) DOI: 10.1038/tp.2012.99

Tuesday, 23 October 2012

The Psychology of Edgar Allan Poe

A paper by psychology undergrad Erica Giammarco offers a look at the mind that gave us The Raven and The Masque of the Red Death: Edgar Allan Poe: A Psychological Profile


Poe lost his mother to tuberculosis at the age of 2; he was then adopted, but his foster mother died young as well. He enrolled at the University of Virginia but became involved in gambling and had to ask his foster father for money; they argued and at the age of 20, Poe was cut off from his family. He married, but his wife suffered frequent illnesses, and died at the age of 25 in 1847; by this time Poe was drinking heavily and he died after collapsing 'drunk and delirious' in 1849.

According to Giammarco:
Poe was described as a mischievous child, playing practical jokes on classmates and teachers... One teacher was quoted as saying that Poe had an "...excitable temperament with a great deal of self-esteem." This grandiose self view would remain consistent throughout Poe’s life; however, Poe was defensive and threatened by negative comments. This is consistent with a narcissistic self-view rather than healthy self-esteem.
Although successful in his studies, he did not have many friends and wrote that school was a "miserable" experience. Classmates stated that he was incredibly defensive and did not allow others to get close...

As Poe aged his health deteriorated and he continued to drink heavily. He was described by coworkers and family as chronically melancholic, acquiring the nickname ‘the man who never smiles’... Poe had a great deal of pride, evident in his refusal to accept money when he and his wife were both sick and unable to work...

An examination of the letters Poe wrote to family reveals that he was a dramatic individual. He often used excessive, theatrical language, poignantly captured in his statement, "I do believe God gave me a spark of genius, but He quenched it in misery"


When describing Poe in terms of the Five-Factor Model of personality we can conclude that he would be high on Neuroticism – evident by the constant nervous anxiety he was said to have, as well as his melancholy and irritability. Poe would also be described as being low in Agreeableness and Conscientiousness since he was argumentative, untrusting, and lacked self-control (i.e. his drinking, his failure to pursue education).
Poe actually crops up several times in the medical literature. Other examples of scientific anthropoelogy include...
ResearchBlogging.orgGiammarco, E. (2013). Edgar Allan Poe: A psychological profile Personality and Individual Differences, 54 (1), 3-6 DOI: 10.1016/j.paid.2012.07.027

Saturday, 20 October 2012

When Replication Goes Bad

How to ensure that results in psychology (and other fields) are replicated has become a popular topic of discussion recently. There's no doubt that many results fail to replicate, and also, that people don't even try to replicate findings as much as they should.


Yet psychologist Gregory Francis warns that replication per se is not always a good thing: Publication bias and the failure of replication in experimental psychology
Among experimental psychologists, successful replication enhances belief in a finding, while a failure to replicate is often interpreted to mean that one of the experiments is flawed. This view is wrong.

Because experimental psychology uses statistics, empirical findings should appear with predictable probabilities. In a misguided effort to demonstrate successful replication of empirical findings and avoid failures to replicate, experimental psychologists sometimes report too many positive results.

Rather than strengthen confidence in an effect, too much successful replication actually indicates publication bias, which invalidates entire sets of experimental findings...

Even populations with strong effects should have some experiments that do not reject the null hypothesis. Such null findings should not be interpreted as failures to replicate, because if the experiments are run properly and reported fully, such nonsignificant
findings are an expected outcome of random sampling... If there are not enough null findings in a set of moderately powered experiments, the experiments were either not run properly or not fully reported. If experiments are not run properly or not reported fully, there is no reason to believe the reported effect is real.
Say you took a pack of playing cards and removed half the red cards. Your pack would now be 2/3rds black, so if you took a random sample of cards, say a poker hand of 5 cards, then you'd expect more blacks than reds (a significant 'effect' of color). But you'd still expect some reds, and some random hands would in fact be entirely red, just by chance. If someone claimed to have drawn 10 random hands and they'd all been mainly black, that would be implausible - "too good".

Francis's approach is a bit like Uri Simonsohn's method for detecting fraudulent data - they both work on the principle that "If it's too good to be true, it's probably false" - but they differ in their specifics, and I believe that we should not conflate fraud with publication bias... so let's not get carried away with the parallels.

Earlier this year, Francis wrote a critical letter about a paper published in PNAS purporting to show that wealthier Americans are less ethical. He argued that the paper's results were "unbelievable" - it reported on the results of seven separate experiments, all of which showed a small, but significant, effect in favour of the hypothesis.

Even if rich people really were meaner, Francis said, the chance of 7/7 experiments being positive is very low: just by chance, you'd expect some of them to show no difference (given that the size of the difference in those seven was low, with a lot of overlap between the groups). Francis suggested that the authors may have run more than seven experiments, and only published the positive ones; the authors denied this in their Letter.

Anyway, in the new paper, Francis expands on this approach in much more detail, drawing from this 2007 paper, and suggests a Bayesian approach that might help mitigate the problem.

ResearchBlogging.orgFrancis G (2012). Publication bias and the failure of replication in experimental psychology. Psychonomic Bulletin and Review PMID: 23055145

Wednesday, 17 October 2012

Brain Mnemonics

Learning your way around the brain is pretty difficult; I spent a long time trying to learn the basics of neuroanatomy... and I bet if you did a pop quiz of neuroscientists the results would be rather embarrassing.

Over time, I've come up with mnemonics to help me remember the trickiest points. Here's some of my favourites.


Gyri and Sulci - which is the ridge, and which the trough? A sulcus sounds like sulk which is what you do when you're feeling low - sulci are the troughs.

Dorsal and ventral - which is lower? Ventral, because the V in ventral is an arrow pointing down.

Axial, Coronal and Sagittal sections - Sagittal is the only kind of cut that could cause Split-brain Syndrome by cutting the brain into two Symmetrical halves.

Axial is the section you'd get if you drew a line across your the face from left to right (and then extend that back), or in other words around your head.

If you have any other good ones, please share them in the comments.

Sunday, 14 October 2012

More on False Positive Neuroimaging

Back in June, I warned that the ever-increasing number of clever methods for analyzing brain imaging data could be a double-edged sword:
Recently, psychologists Joseph Simmons, Leif Nelson and Uri Simonsohn made waves when they published a provocative article called False-Positive Psychology - Undisclosed Flexibility in Data Collection and Analysis Allows Presenting Anything as Significant.
It explained how there are so many possible ways to gather and analyze the results of a  simple psychology experiment that, even if there's nothing interesting really happening, it'll be possible to find some "significant" positive results purely by chance...
The problem's not just seen in psychology however, and I'm concerned that it's especially dangerous in modern neuroimaging research.
In a comment on that post, The Neurocritic pointed out that Michigan PhD student Joshua Carp had put forward the same argument in a conference presentation, several months previously.

Now Carp's published a paper on the topic: On the plurality of (methodological) worlds: estimating the analytic flexibility of fMRI experiments. It's free to access, so check it out.

Whereas I just talked the talk by listing lots of possible ways in which you could analyze a given set of data, Carp walked the walk, and actually did loads of analyses. He took a single dataset, the results of a simple experiment and looked at it in almost 7000 different ways. Each set of results was then thresholded to correct for multiple comparisons in 5 ways, for a grand total of 35,000 outputs.

The variants he considered ranged from how much smoothing to apply, to how to correct for head motion, and many more.

What happened? In a nutshell, the different options made a difference - and the variability was the largest in parts of the brain that were most activated (the "blobs" that lit up). In other words, analytic flexibility makes the most difference in the most interesting places. See the picture at the top.

The location of the maximum peak activation also varied. This is not unexpected, and not, in itself, that worrying - the great majority of the peaks clustered in a few small areas. However, it underlines that different options really can make a difference.

Carp concludes:
Nearly every voxel in the brain showed significant activation under at least one analysis pipeline. In other words, a sufficiently persistent researcher determined to find significant activation in virtually any brain region is quite likely to succeed...

If investigators apply several analysis pipelines to an experiment, and only report the analyses that support their hypotheses, then the prevalence of false positive results in the literature may far exceed the nominal rate. However, analytic flexibility only translates into elevated false positive rates when combined with selective analysis reporting. If researchers reported the results of all analysis pipelines used in their studies, then it would not be problematic.

To the author’s knowledge, there is no evidence that fMRI researchers actually engage in selective analysis reporting. But researchers in other fields do appear to pursue this strategy.
In my experience, fMRI researchers are actually fairly conservative in terms of using different analyses, and certainly I doubt anyone has ever run thousands of them just to get the result they want and I'd estimate that most published findings are not the result of more than a handful of 'attempts' at most.

However it's a serious concern that it could happen, and importantly it's getting ever-easier to do this, with the continuing increase in computer power making running an analysis quicker and cheaper than ever. As to what to do about it, Carp makes several suggestions, and here's one I made earlier...

ResearchBlogging.orgJoshua Carp (2012). On the plurality of (methodological) worlds: estimating the analytic flexibility of fMRI experiments Front. Neurosci. DOI: 10.3389/fnins.2012.00149

Saturday, 13 October 2012

A New Theory of Psychosis?

A team of British neuroscientists led by the (in)famous David Nutt says that magic mushrooms offer a new theory of psychosis: Functional Connectivity Measures After Psilocybin Inform a Novel Hypothesis of Early Psychosis


It's a reanalysis of a study from earlier this year, which got quite a lot of attention, in which 15 volunteers were injected with psilocybin - the major active hallucinogenic ingredient in 'magic mushrooms' - during an fMRI scan.

In a nutshell, the rather interesting proposal in the new paper is that psilocybin may cause mind-altering effects by blurring the difference between the brain networks responsible for 'internal' and 'external' thought.

Activity in the internal "default mode network" (DMN) is generally anti-correlated with the "task-positive network" (TPN) - when one is higher, the other's lower. The DMN is active when you're not doing much - hence 'default' while the TPN comes online when you're engaged in a particular mental activity.

Nutt's team say, however, that their functional connectivity fMRI data show that after psilocybin, activity in these two networks becomes positively correlated - an unusual pattern. They write:
Increased DMN-TPN coupling has been found in people at high risk of psychosis and an inability to distinguish between one’s internal world and the external environment, sometimes referred to as “disturbed ego boundaries,” is a hallmark of early psychoses and the psychedelic state.
One of our volunteers reported the following after psilocybin: “It was quite difficult at times to know where I ended and where I melted into everything around me.”
To be honest I'd need to see a replication before I put too much faith in this, because this kind of post-hoc reanalysis of fMRI data is very flexible and therefore prone to false positives, but it's an interesting idea, and at least it provides a clear theory for further research.

ResearchBlogging.orgCarhart-Harris RL, Leech R, Erritzoe D, Williams TM, Stone JM, Evans J, Sharp DJ, Feilding A, Wise RG, and Nutt DJ (2012). Functional Connectivity Measures After Psilocybin Inform a Novel Hypothesis of Early Psychosis. Schizophrenia bulletin PMID: 23044373

Sunday, 7 October 2012

Getting The Position Right For EEG

In science, it's often the most 'boring', easily overlooked factors that determine whether an experiment succeeds or fails.

A new paper reveals strong effects of body posture on brain electrical activity: Subject position affects EEG magnitudes. Just lying face-up as opposed to face-down can powerfully affect the signal measured using electroencephalography (EEG), according to Justin Rice and colleagues of New York.

Here's why: EEG uses electrodes, placed on the scalp, to measure the electrical potentials produced by brain firing.

The signal recorded depends, however, not just on the brain activity but also on the quality of the electrical conduction between the brain and the scalp: the signals have to travel through the fluids surrounding the brain, then the skull, and finally the skin, before they're detected.

EEG users sometimes think of brain-scalp conductivity as a fixed factor, that they can't control and don't have to worry about. However, Rice et al point out that the position of the brain shifts within the skull depending upon your posture.

This is because there's a bit of extra room in there, leaving space for the brain to "bounce around" a little within its fluid cavity. If you're lying on your back, the brain will lie closer to the back of the skull; if you're on your front, it'll be further forward, and so on. So the fluid layer between brain and skull will be corresponding thinner, or thicker.

In a healthy brain the change is only about 1 mm, but the fluid layer's only 3 mm at most, so that's a big change.

Others have recognized this problem before, but Rice et al's data are the clearest evidence yet that posture is a major factor. They showed that subjects lying on their back (supine) showed significantly stronger activity over the back of the brain - which makes sense, as it brings the brain closer to the electrodes. Lying face down (prone) made activity weaker and sitting was in between.

Interestingly - and worryingly - the effect was stronger depending upon the kind of activity being measured. For most kinds of brain activity it was about 40% higher but for gamma waves - the hottest thing in EEG right now - it was almost 80%.

So gamma band activity is especially sensitive to posture, and that raises the worrying possibility that even slight differences in head position between individuals could account for 'differences' in gamma power recorded, for example in studies comparing neurological patients and healthy controls; if the controls are sitting up straight while the patients are slouching back... the patients would seem to have more gamma.

ResearchBlogging.orgRice JK, Rorden C, Little JS, and Parra LC (2012). Subject position affects EEG magnitudes. NeuroImage PMID: 23006805

Friday, 5 October 2012

Are Gay Men Happier?

A neat little study from UCLA psychologists Francisco J. Sánchez and colleagues examines the mental health of homosexual men using a unique identical twin design.

The paper kicks off with a remarkably lucid introduction:
Men would rather drive around lost than stop and ask for directions. Although this is a gross stereotype, the notion that men should be self-sufficient and able to solve their own problems is a dominant ideal within traditional views of masculinity... men who rigidly adhere to such ideals may harm their own health if they avoid seeking help when they need it.
In general it reads more like a blog post than an academic paper, which is great. If all papers (and especially social science ones) were written this way, I would be a much happier man. Speaking of happy men...

The authors' basic idea is that many men, wishing to appear 'manly', don't talk about or get help for their problems, especially psychological issues: boys don't cry, and men certainly don't. However, the authors argue that gay men, generally less encumbered by traditional masculinity, may be an exception to this rule.

So they took 38 pairs of male 'identical' twins, who grew up together, but who weren't quite identical: one of each was gay, and one straight. By controlling for most genetic and environmental factors, these twins provide a kind of natural experimental test of the effects of homosexuality per se. Not a perfect one, but about as good as we're going to get.

In accordance with the authors' predictions, gay men were indeed more open to seeking psychological help.

But unexpectedly, they were actually less likely to report experiencing psychological distress (on this scale). That's surprising, given several previous reports of higher rates of mental illness in homosexuals, which has been dubbed 'velvet rage'.

Sánchez et al's data suggest that gay men may be, er, more gay (...the other kind), and that their increased rates of diagnosed mental illness are a product of their greater willingness to seek help: maybe straight men are just in denial.

But there's a lot of caveats here. It's a small study, based purely on self report measures, and the gay twins were compared to their own straight twins, but those twins are quite possibly not typical of straight men in general. It might also be that having a straight twin makes life easier for gay men. Still, it's an interesting set of data.

ResearchBlogging.orgSánchez FJ, Bocklandt S, and Vilain E (2012). The Relationship Between Help-Seeking Attitudes and Masculine Norms Among Monozygotic Male Twins Discordant for Sexual Orientation. Health Psychology PMID: 23025300

Wednesday, 3 October 2012

The Two Problems With Science


There's lots of concern at the moment over mistakes, misconduct and misbehaviour in science.
This concern is a good thing. There are serious, systemic problems with modern science as I and many others have long argued.

However, I worry that much of the recent discussion has failed to distinguish between two fundamentally distinct problems. On the one hand, we have outright fraud - i.e. making up data, or otherwise lying, breaking the basic rules of science.

On the other hand we have questionable practices such as: publication bias, p-value fishing, the File Drawer, sample size peeking, post-hoc storytelling, and all of the other dark arts that can lead to false positive science. These are permissible, even encouraged, by the current rules of doing and publishing science.

These two problems are similar in some ways - they're both "bad science", they both lead to failures to replicate, etc. - but in underlying essence they're very different, so much so that I'm not sure they can be usefully discussed in the same breath.

Fraud and questionable practices are different in terms of their harms. Fraud is a more serious act and it causes local harm, introducing major errors into the record. But in terms of its overall effects, I believe questionable practices are worse, as they systematically distort science: ensuring that, in some cases, it is difficult to publish anything but errors.

Fraud and questionable practices call for different solutions. Broadly speaking, fraudsters break the rules, so to stop them we need to enforce those rules, via deterrence, detection, and punishment - like with any criminal act. With questionable practices, it's the opposite: here the problem is the rules (or the lack of them), and the solution is to reform the system.

It's been suggested that fraud and questionable practices share a common cause in the "pressure to publish", the "publishing environment", the "culture" of modern science etc. But while this is a good explanation for questionable practices, I don't think this can explain fraud, any more than, say, the desire for money can explain theft.

Yes, thieves desire money, and yes they steal in order to get money, but everyone else wants money as well, yet most of us don't steal, so that's not an explanation. Frauds fake data to produce publications. But all scientists are under pressure to produce good publications and they always have been - which is why fraud is not new - what's changed recently is the criteria for a 'good' publication.

Now in retrospect, I blurred these distinctions somewhat with my own 9 Circles of Scientific Hell, in which I placed 6 questionable practices and 2 forms of misconduct on the same scale of "sinfulness". In fact there are two distinct hierarchies. In my defence though, that was a cartoon.

I think finance offers a great analogy here.


In finance, you have some people who break the rules. Bernie Madoff is the current poster boy for this. Such people harm others by outright criminal acts. But then we have the people who play by the rules, and still cause harm. The global financial crisis was in essence caused by all of the major American banks going all-in on a bet, and losing. Yet no-one broke the rules: the regulations allowed banks to gamble. The problem was not rule-breaking, but the rules (or lack thereof).

Here's the curious thing: the financial crisis did more harm than Madoff's scam, even though what Madoff did - theft by fraud - was more immoral than what the bankers did - gambling unwisely.

That's confusing to our ethical sense and our emotions (who should we feel more angry at? Who's 'worse'?) but it's really no surprise: precisely because what the banks did was above board, everyone did it so the damage was huge. If it had been illegal for banks to gamble all their money at once, individual banks might still have broken that rule, locally, but it's unlikely that the system would have been threatened.

Maybe you can see where I'm going with this: everyone following bad rules is often worse than individuals breaking good rules.

Science has its share of fraud. Hauser, Smeesters, Fujii - they broke good rules against such deceit. They are the Bernie Madoffs of science. But then there's 'questionable practices' like publication bias, p-value fishing, the File Drawer, and all the rest, which are allowed, but which are universally acknowledged to be bad for science. Scientists using these dark arts (and I don't know any who never do) may be the Lehman Brothers of science.