The only sure way is to study it in detail and know all the technical ins and outs. But good ideas and bad ideas behave differently over time, and this can provide clues as to which ones are solid; useful if you're a non-expert trying to evaluate a field, or a junior researcher looking for a career.
Today's ideas are the basis for tomorrow's experiments. A good idea will lead to experiments which provide interesting results, generating new ideas, which will lead to more experiments, and so on.
Before long, it will be taken as granted that it's true, because so many successful studies assumed it was. The mark of a really good idea is not that it's always being tested and found to be true; it's that it's an unstated assumption of studies which could only work if it were true. Good ideas grow onwards and upwards, in an expanding tree, with each exciting new discovery becoming the boring background of the next generation.
Astronomers don't go around testing whether light travels at a finite speed as opposed to an infinite one; rather, if it were infinite, their whole set-up would fail.
Bad ideas generate experiments too, but they don't work out. The assumptions are wrong. You try to explain why something happens, and you find that it doesn't happen at all. Or you come up with an "explanation", but next time, someone comes along and finds evidence suggesting the "true" explanation is the exact opposite.
Unfortunately, some bad ideas stick around, for political or historical reasons or just because people are lazy. What tends to happen is that these ideas are, ironically, more "productive" than good ideas: they are always giving rise to new hypotheses. It's just that these lines of research peter out eventually, meaning that new ones have to take their place.
As an example of a bad idea, take the theory that "vaccines cause autism". This hypothesis is, in itself, impossible to test: it's too vague. Which vaccines? How do they cause autism? What kind of autism? In which people? How often?
The basic idea that some vaccines, somewhere, somehow, cause some autism, has been very productive. It's given rise to a great many, testable, ideas. But every one which has been tested has proven false.
First there was the idea that the MMR vaccine causes autism, linked to a "leaky gut" or "autistic enterocolitis". It doesn't, and it's not linked to that. Then along came the idea that actually it's mercury preservatives in vaccines that cause autism. It doesn't. No problem - maybe it's aluminium? Or maybe it's just the Hep B vaccine? And so on.
At every turn, it's back to square one after a few years, and a new idea is proposed. "We know this is true; now we just need to work out why and how...". Except that turns out to be tricky. Hmm. Maybe, if you keep ending up back at square one, you ought to find a new square to start from.
9 comments:
The autism-vaccines theory is an especially poignant example: The original publication has been retracted, and yet this whole idea has had an immense cultural impact and still enjoys a loyal following.
Another bad idea? The "chemical imbalance" theory of depression.
Something I find off is when scientific papers vaguely justify botchiness through historical background or some obscure reasoning. Since there's so many specialized scopes out there it seems difficult to standardize what's acceptable or not.
I noticed a classic science paper appears highly logical but also wows with coherent sets of ideas within that field. They can be justified mathematically, become law, vaccines, aerodynamically or technologically feasible etc.
I noticed today there's so much report on observations for the sake of observation with not much thought. I mean science as observation duh, but what have you discovered? I'm sure you can train a monkey to observe as well.
I kind of blame the rigorous hypotheses method systems. It just seems so ridiculously belated in college and journals.
As a graduate student, how would you know which node to pick? I'm curious about the origin of the tree diagrams -- could you give more examples of how to differentiate between a good assumption and a bad assumption?
To zero in on one point you made, I agree that once an assumption has gained footing, it can stick around for a long time. Veri sort of touched upon the point that within a field, a coherent set of ideas can become an accepted doctrine. I imagine it's harder for a researcher within the field to get funding for genuinely innovative work that links various subfields, without paying homage to whatever the current trends happen to be in their own specialization.
I think even shaky assumptions can be the springboard for innovation when someone comes along and connects the dots in the right way. Again, that's probably not easy to do within one sub-specialized area of study, though. One poster mentioned the chemical imbalance theory of depression. It's not necessarily that the idea itself is bad, but the momentum (particularly the marketing hype) caused people to over-focus on SSRIs. One branch kinda sorta leads to another branch. Maybe. If we keep narrowing the focus.
Speaking in very very abstract terms, I think that's the problem. If you happen to see a common thread between fields (from primary research papers, without referring to one hypothetical construct after another), you might be onto something, but you're taking a chance going out on that limb.
veri, the trouble is that classical science papers only really work within a reductionist framework - for some research questions and fields, that's no good.
Work based on systems theory is not somehow inferior.
Anonymous #1: Right. But the idea has changed, at first it was specifically MMR linked to autism w/ GI problems due to the measles virus, a theory which is no longer popular even with the anti-vaccine crowd. But other ideas take its place.
veri: Quite right. The thing is, even great scientists don't come up with great ideas all the time, but in order to succeed in science nowadays you need to publish and the only way to do that is to publish all your observations, even if you have no idea what they mean.
RJ: Knowing which node to pick is hard, but I would say that you should focus on stuff which has a good pedigree: if it's grown out of work from say 5 years ago which has become widely accepted (and that, in turn, was based on good work from 10 or 15 years ago...) What you should avoid are fields in which there are loads of people all trying to solve one problem from 20 or 30 years ago; although that's only a general rule. If you strike lucky you might solve that problem and win a Nobel prize... but it's unlikely.
RL:
Believe me, the "chemical imbalance" theory of depression is NOT a node worth watering. But it can sure stand pruning.
MW, I see what you're saying and I agree some papers do tend to seem waay too idiosyncratic. I guess I was referring to the distribution of ideas. Take for example a politician. A new politician promises the world, a revolution etc. but a veteran politician considered a revolutionary among peers is able to deliver that revolution in the details.
They inspire because they can come up with workeable ideas in specific policies, might be subtle but makes a difference not just for that but across other disciplines as well. Of course s/he too probably began his/her journey calling for a revolution.
I don't regard any theory as inferior, but the futility of not being able to accommodate other modes of processes , systems, paradigms etc because of money, approval etc. should not hinder the thought process itself. Of course there are other ways to expand on horizons to what what aside from college and journals.
Post a Comment