What Is Science?

The other day I was in a discussion about what "science" is. I've written about this before but this debate got me thinking about it again and I thought I'd set out what I think in more detail.

I wasn't sure how best to structure this so I'm going to start with my main claim, followed by a Q and A bit. The Q's are not intended to be straw-men or caricatures, they're questions I've asked myself in the course of thinking about this.

My Claim: "Science" is just the process of looking at the world and thinking about the evidence in an effort to understand it. It's not a special form of knowledge, scientists don't use a special 'scientific method' - scientists just look and think about things. They may use special equipment and techniques, but in essence it's no different to what we all do every day. As such it makes no sense to talk about the 'limits of science' or 'what science can't tell us', unless by that we mean the limits of human knowledge itself, because science just is knowledge.

Q: "But if science were just observation, everyone would be a scientist and it becomes meaningless." - No: for the same reason that not everyone is a poet, even though anyone can write a poem.

Science is observation informed by previous scientific findings i.e. it is expert observation. Anyone can, say, look at the stars but this doesn't make them an astronomer. Astronomy is in essence just looking at the stars and thinking about them - in a broad sense - but to contribute to astronomy, you first need to know the relevant background, which few except astronomers do.

Likewise, anyone can write a poem, but few of us can make a living out of it.

Q: "OK, but still, if science is just observation, then all forms of knowledge are science." - This is a tricky one, but the answer is crucial to understanding my point.

A few hundred years ago, the word "science" in English did indeed just mean "knowledge". However, more recently, it has come to mean a particular subset of knowledge: roughly, it today includes physics + chemistry + biology. Maybe some others.

We lump these three (or more) things together and call the lump "science". But this lump is more or less arbitrary. Physics + chemistry + biology don't share a special essence, which sets them apart from other kinds of knowledge (other 'sciences' in the older sense.)

So I'm not saying that all knowledge is "science" in the modern sense. The modern word "science" only includes a limited portion of knowledge. But I am saying that the rest of knowledge is essentially no different from science, because "science" is just an arbitrary subset of knowledge.

Here's a picture of what I mean:

Or here's an analogy. The color spectrum has infinite different shades. We conventionally divide it up into "red", "orange", "yellow" etc. and that's fine for most purposes. But there is no essential difference between "orange" and "yellow" and no clear dividing line: they are just collections of shades.

Science is a colour of knowledge. It's not a true kind.

Q: "But if the difference between science and knowledge is arbitrary, are you saying that The Scientific Method is the only way to knowledge?" Not at all. I don't think 'The Scientific Method' exists.

This follows from the fact that "science" is an arbitrary lump. Scientists are a diverse bunch and they use many different methods. Theoretical physicists, for example, use methods which are very close to those of mathematicians - who are not 'scientists' by most definitions. Zoologists use others, and you can get by in (most) of zoology without knowing any math at all. And so on.

In fact, there's almost as much difference between branches of "the same science" as there is between sciences. Just like "science", "biology" is a lump of diverse things, although not quite as arbitrary a lump.

Outside science, people use all kinds of methods as well. Historians have their set of methods, economists have others, all tailored to the particular demands of the case. That's exactly how it should be - all knowledge comes from observation and, to observe different things, you need different methods.

There are many different kinds of facts, but a fact is a fact, whether it's a scientific fact, a historical fact, or just an everyday fact. "Shakespear wrote Hamlet" is just as true as "The earth orbits the sun" is just as true as "It's raining" (if it is, in fact, raining.) The facts of history are just as true as the facts of biology.

But there is a grain of truth in the question here: I am saying that observation is the only way to knowledge.

Q: "That's very simplistic. Are you saying that we can only know what we can see and measure?" - No because I'm using "observe" in the broadest sense here, to include things like noticing, sensing, feeling, seeing, hearing, being told that, reading about...

You observe that it's raining: maybe you look out the window, maybe you hear the rain on the roof, maybe someone comes in from outside dripping wet. You observe that you feel hungry. You observe that Obama is president (even if you've never actually seen him) by watching the news. Etc.

"Observe" is in many cases an awkward word and I'm not saying we should use it in all those cases. My point is that these are all ways of finding stuff out about the world. In that broad sense, I think all knowledge comes from observation - although maybe indirectly, through thinking about observations; thoughtless observation is almost as unhelpful as purely abstract speculation.

Of course, it's easier said than done: very often it's not clear what we've observed (is it real or some kind of trick, mistake, illusion?), what it means, whether it matters, or even what the question is. This is why there's always room for debate, controversy and doubt, at least at first, before all the observations have been made.

Q: "But what you're saying is merely trite and obvious: 'science is based on observation' 'there is no one single scientific method', well, duh!" - From a certain perspective, they are obvious, but if you think so, then you ought to agree with everything else I've written here. They're all connected, you can't take some and leave the rest.

Autoerotic Asphyxiation For Science

The death of an autoerotic asphyxiation fan ended up providing science with some valuable observations of what happens during choking

On Twitter recently, I've been highlighting some really bad ideas courtesy of the medical literature. From injecting vaseline into your own penis, to pumping compressed air up your rectum for a joke, people have tried it and they've ended up on PubMed as a result.

But a recent paper details an extreme case. Canadian medics Anny Sauvageau and colleagues report on the life and death of a 35 year old Hispanic man who, we're told,

filmed several of his autoerotic asphyxia sessions. After his accidental hanging death, several films were found at the scene, including 3 filmed nonlethal ligature strangulations.

In these 3 ligature strangulations, the man is sitting on a chair. A pair of pajama pants is rolled once around his neck, with the extremities of the pants falling down on each side of his chest. The man is pulling the extremities of the pants with both hands to apply compression on his neck. After losing consciousness, he ceases to pull on the ligature, and the pants slowly loosen around the neck. A few seconds later, he regains consciousness and gets up from the chair.

The videos - which before you go and look, are not included as Supplementary Materials - are of scientific interest, because they offer a rare glimpse into what happens during human strangulation.

You might think that given that people have been strangling each other for thousands of years, it would be a well-understood phenomenon, but in fact it's very difficult to study. Most of the available evidence comes from video recordings of execution by hanging, but there are only perhaps 10 of these records on file. Although there's also been an interesting line of research involving judo, in which judo masters apply choke holds to willing volunteers. These include one remarkable paper in which brain activity was recorded before, during and after.

Anyway, for what it's worth, this is what seems to happen during strangulation, including hanging:

1. Loss of consciousness (10 +- 3 seconds).
2. Convulsions: (14 +- 3 seconds).
3. Decerebrate rigidity: (19 +- 5 seconds)
4. Start of very deep respiratory attempt (19 +- 5 seconds)
5. Decorticate rigidity (38 +- 15 seconds)
6. Loss of muscle tone (1 minute 17 seconds +- 25 seconds)
7. End of the very deep respiratory attempt (1 minute 51 seconds +- 30 seconds)
8. Last muscle movement (4 minutes 12 seconds +- 2 minutes 29 seconds)

The man in this case took himself to Stage 2 before recovering.

Until the last time he tried it, when he didn't.

Sauvageau A, Ambrosi C, and Kelly S (2012). Three Nonlethal Ligature Strangulations Filmed by an Autoerotic Practitioner: Comparison of Early Agonal Responses in Strangulation by Ligature, Hanging, and Manual Strangulation. The American journal of forensic medicine and pathology PMID: 22922552

Beyond Self-Report

If you want to learn about someone, should you ask them?

Two bits of research published recently cast doubt on the validity of self-report as a tool in psychology and psychiatry. The first found that teens who reported that they suffered from bullying experienced more mild 'psychotic-like' symptoms. That correlation would be consistent with the idea that these symptoms arise as a response to stress.

However - the same study found that there was absolutely no correlation between peer ratings of whether someone was bullied, and their psychotic symptoms. Only self-report was associated.

The second study looked at whether a crisis intervention program - intensive mental health care - helped people who'd recently attempted suicide. The results showed that, compared to a control condition, suicidal patients who got crisis care self-reported fewer subsequent suicide attempts.

But a trawl of hospital records painted exactly the opposite picture - the intervention group were more likely to end up in hospital with a second attempt.

Remarkably few studies in psychology and psychiatry compare self-report to other measures of behaviour. This is because self-report is typically a lot easier. If these papers are anything to go by, however, this is a serious limitation. Self-report can be radically at odds with other measures of behaviour.

Which raises the question - who's right? Are the self-reports right, when they clash with other sources? I don't think there's an easy answer. In the bullying case, it might be that self-report is more accurate, because the peers doing the peer-report are the bullies. In the suicide study, maybe the self-report was more accurate, because the patients knew about attempts that never made it to hospital.

But on the other hand you could argue exactly the opposite. Maybe the self-reports of bullying just reflected whether the kids thought their classmates liked them. Maybe the patients were ashamed to admit that they'd reattempted suicide even though they'd got all this special crisis care. It's hard to tell.

One thing's clear though: self-report is not the whole story.

Gromann PM, Goossens FA, Olthof T, Pronk J, and Krabbendam L (2012). Self-perception but not peer reputation of bullying victimization is associated with non-clinical psychotic experiences in adolescents. Psychological medicine, 1-7 PMID: 22895003 

Morthorst B, Krogh J, Erlangsen A, Alberdi F, and Nordentoft M (2012). Effect of assertive outreach after suicide attempt in the AID (assertive intervention for deliberate self harm) trial: randomised controlled trial. BMJ (Clinical research ed.), 345 PMID: 22915730

Replication Alone Is Not Enough

Psychology has lately been hit by high-profile fraud scandals, and broader concerns over questionable research practices. Now the Society for Personality and Social Psychology (SPSP) has released a statement on "Responsible Conduct", and a task force has produced a report.

This is a start, and the SPSP is to be commended for facing up these problems (which affect many other fields) relatively early. However, neither of their documents contains much meat in my view.

Point One on the task force report is that "Replication is the key to building our science" and they suggest a "web site for depositing replications and failures to replicate" - but don't mention that various enterprising researchers have already made one. Nor do they tip their hats to the Open Science Initiative addressing just this issue. This makes me worried that they're planning to reinvent the wheel.

More fundamentally I disagree that replication is key to psychology or any field. Our goal should be replicability. Failure to replicate findings is a symptom of problems with those original findings, rather than being a problem in and of itself. Good results replicate; we want better results to be published.

In other words, we should strike at the root cause of invalid research, namely, the perverse incentives towards publishing as many eye-catching positive results with p values below 0.05 as possible by any means necessary. P-value fishing, selective reporting, post-hoc "prior hypotheses" and other questionable practices are a large part of what make unreplicable results.

We should encourage replication, but it's no panacea.

An overemphasis on replication, without addressing the incentives, could actually harm science. It could lead to scientists spending all their time worrying about the political drama of who's replicating who and why, and which questionable practices they can use to replicate their friends' data - rather than actually doing science.

This is why we shouldn't be satisfied with any reform effort that puts replication before replicability. If you can fudge a result, you can fudge the data a replication. How to fight questionable practices is another question but I've proposed reforms that I think would work, namely pre-registration of hypotheses, methods, and statistical analyses. Others have their own ideas.

A lesson from clinical medicine here. Clinical trials of new drugs adopted pre-registration, but only after they tried replication and it didn't work. Pharmaceutical regulators have long required multiple demonstrations of drug efficacy. One trial was not enough. Sounds good - but the problem was that drug companies just did lots of trials and analyses, picked the positive ones, and used them.

So in summary: replication is important, and we don't do enough of it, but replication alone is not enough to fix psychology.

Neuroscience: Solving The Hard-On Problem

Take a good look at this, fellas:

It might not look like much, but this is science's very first glimpse of something rather close to the hearts of most men.

These images show nerve activation in the spinal cord during sexual arousal. Using functional magnetic resonance imaging (fMRI), best known as a way of recording brain activity, applied to the spine, Canadian researchers Kozyrev et al were able to record the changes associated with, well, stimulation. Here's the paper: Neural correlates of sexual arousal in the spinal cords of able-bodied men.

The methods sound like a real riot. Ten guys were scanned while being exposed to two different kinds of stimulation. First, 'audiovisual'

Films, depicting heterosexual couples engaged in sex play, vaginal sexual intercourse, and oral sex, were presented in two 5-min blocks, separated by a baseline condition (blank screen) for 3 min... The selected films were chosen by the investigators and were all very highly rated (on a scale of 1 to 10) by independent viewers. Only films that received a minimum score of 8 out of 10 were shown to participants.

They don't say who these independent grumble-rating viewers were, but I'm, er, told that some porn sites these days have user ratings next to each clip. So it's possible that they used those and that the raters were, quite literally, just some bunch of wankers.

As for the manual stimulation...

All participants manually performed stimulation of the penis. Participants received written instructions superimposed onto a rear-projection screen, indicating whether to rest or to perform penile self-stimulation... there were two equal blocks of 1.5 min of stimulation that were separated by a 1.5-min block of rest... a shorter duration was selected for this part of the study in order to avoid overstimulation of the penis and/or ejaculation.

Good thinking. That could have been messy.

The results of the study largely confirmed previous work, but it's an important step forward because until now, no-one had measured arousal-related spinal activation directly in healthy humans; the evidence came from recordings in animals and spinal injury patients.

Clearly there's more work to be done here. The images are rather crude. That grey oval is meant to look like this:

Still, it's early days. Spinal cord fMRI is not a new technique, having first been performed in 1999, but it's not received nearly as much research interest as the brain variety.

Kozyrev N, Figley CR, Alexander MS, Richards JS, Bosma RL, & Stroman PW (2012). Neural correlates of sexual arousal in the spinal cords of able-bodied men: a spinal FMRI investigation. Journal of sex & marital therapy, 38 (5), 418-35 PMID: 22900624