Using a BS detector on popular science reporting


Ben Goldacre's latest "Bad Science" column for the Guardian is "How to read a paper," a great editorial explaining how to critically evaluate scientific claims that are printed in the newspaper:

Our next case takes more elaborate checking, since it involves an experiment and its interpretation. Scientists at Lancaster University, say the Daily Mail and the BBC, have devised an amazing piece of paedophile identification software. It reads your messages and decides if the person you're chatting to on the internet is another young person, or an adult who is pretending to be young.

This is a tricky problem to solve on a handheld device, or indeed anywhere. There is a press release on the Lancaster University website explaining that this device has been studied and found to work. I asked for details. The methods and results of this study are secret. No paper has been submitted for publication.

So actually there's no complicated interpretation problem here: nobody can know what these scientists measured, how they measured it, what the numbers were like, how closely the experiment mirrored a real world situation, or anything at all. When the Raelian cult said they'd cloned a baby, but we weren't allowed to see it, nobody took them seriously. Until someone's willing to tell me what they measured and how they measured it, they might as well be Raelians.

A useful, but admittedly more blunt heuristic might be: "If it's in the Daily Mail, it's probably not true."

How to read a paper

(Image: Mail Online screengrab)