Security and the statistics of rare events
My latest Guardian column just went live, on the security risks of the failure to grasp the statistics of rare events:
This is the same calculus that allows the fear of terrorism to take away our liberty: the statistically super-rare terrorist attacks present, on average, a much lower risk to our health, safety and person than, say, depriving us of our liquid medications, or of requiring us to leave our bags unlocked in flight so that sticky-fingered handlers can make off with our laptops and financial data and valuables.LinkThe everyday threat of having our goods stolen, our ability to travel and earn our livings curtailed, and our personal information harvested by every junior terrorist fighter who wants to see your ID before letting you do anything is overshadowed by the one-in-a-billion confluence of someone with terrorist goals, the means to accomplish them, and the intelligence to bring them off (hint: you can't really blow up an airplane with hair-gel and iPods).
the latest
latest episodes
Great Article! One of our best yet. I often use the arguments of statistics and logic with ny wife, but she still seems to live in fear a bit too much for liking. Any advice on how to use these points to win over friends and family?
How about accepting the fact that other people can have view-points that are different than your own and that it is out of your control?
statistics are like guns. without going into much detail, there's a very readable book by lee clarke on rare events which i read for my research:
http://www.amazon.com/Worst-Cases-Catastrophe-Popular-Imagination/dp/0226108597/ref=pd_bbs_1?ie=UTF8&s=books&qid=1211316749&sr=8-1
#2: How about accepting the fact that other people have view-points that are different from YOURS?
Not everyone believes the world is a hopeless place where one man can't make a difference.
#2: Is that a general comment about how we should all live our lives? If everyone's viewpoints are "out of your control," what would be the point in ever discussing anything at all?
The Homeland Security laws, and the rules about flying being enacted to "fight terrorism" have nothing to do with keeping us safe. Of course, if they were about protecting citizens, then the whole lot is a big, huge, inefficient, inexplicable, utterly unworkable joke. Everybody knows it, and the internet is full of people patiently explaining why these laws are patently ridiculous. But what if it's NOT about protecting citizens, but controlling them? Ah, then they make sense- then, they perform their tasks admirably, and lead to ever tightening reigns of control.
It's a good article Cory, but you're missing the point entirely. The people that are enacting these laws are not afraid of the super-rare events. They are using those as an excuse, just like every other Fascist government dreamed up a bogeyman to scare the populace with, to give them excuses to take control away from the citizens. The nazis did it, perfectly legally, just like is happening now.
Unless you DO understand the point, but you are trying to break it gently to those who don't understand yet, without sounding like a basketcase or a conspiracy theorist?
Ah well I don't have any reason to care if people think I'm a nutcase, I'll just say it.
Nice article. Now I have something to email my parents and in-laws when trying to talk to them about security theater and how our rights are being eroded without, as LilOrfnAnnie put it 'sounding like a basketcase or a conspiracy theorist.' (They usually shrug and mumble something like, "What can you do?")
Sad to say, but it sometimes seems Americans WANT to be scared. If we don't have an enemy 'out there', we'd have to face all the problems eating away the country from the inside.
Xenu @ #4 -- one person can make a difference, true, but the time has to be right for that action and it has to be done by the right person. You can't draw down lightning if there isn't enough charge in the air. We're not there, yet.
#2 -- kinda harsh and out of nowhere. Isn't there a Little Brother grouse-fest you can join? (same goes for you, @ #16)
The "teach people statistics" bit is never going to fly. I mean, what are politicians going to do with themselves in a world where that project has been successfully carried out?
I had to take a couple statistics courses to get my math degree, but I still didn't follow that paragraph. Or maybe that last bit is a misprint?
Maybe I'm just due for another caffeine shot.
Statistically, I think now that I've read this article, I'm 42% more likely to have the party V& pull up out front...
#8 (soon to disappear),
Tell Mom to bring you a snack. You're getting cranky.
I have seen quite a few people who are afraid of the whackos in the world (what most people call terrorists), but yet at the same time they will ride in a car without a seatbelt, smoke cigarettes, overeat, and do other similar actions. They fear losing their life or health to a whacko, but they don't seem interested in taking actions that will very likely provide much greater protection and benefits.
(same goes for you, @ #16)
That should have been (same goes for you, @ #08)
apologies for flicking the nose of #16 before it's even posted.
Why does BB have to be so full of trolling? People complaining about free speech on a moderated forum is so very early Internet.
#11,
Yeah, that sentence is a little misleading. It's wrong 9999 out of 10,000 of the subset of the one million people. It's kind of confusingly written.
I'm not sure if speach is free, I don't follow that market, but yes, public *speech* is free.
Don't forget though that BoingBoing is owned by people, and isn't actually public. Everything on here is there by those people's whim. Go cry about it on a street corner for maximum free speech.
Why do the wannabe radical idiots always misunderstand/ignore this point?
"please stop it with the L B posts"? I must be dense. I don't get the reference.
I have no idea what this person is going on about. Must have been someone who was banned for being a jerk, and is posting under another identity.
#15,
My nose!
The BB comment system really just needs a reply button. Even unthreaded you can do it the way that gawker sites do it. It works out nicely.
# 14 (about to implode),
There's free speech. You can say whatever you want to your inflatable girlfriend.
@#11
math degree and ya didn't get it eh... erm can i ask where it was you were at college... just kidding ;)
here it is:
there is a disease that affects 1 in a million people.
that means that only 1 person out of a sample of a million have it.
you have a test that is 99% accurate.
that means that if you test all 1million people, there is still going to be 1% that get the wrong result
i.e. that means that 10,000 people are going to get a notice that tells them they have the disease even though they don't have it.
strictly speaking cory could have said "10,000 times out of 10,001", rather than "9,999 out of 10,000" but you get the point; because of 10,001 people who got a positive result, 10,000 of them got the wrong result.
Remember about a decade back when the Food Crisis of the Day was possibly-carcinogenic Alar on apples? Very remote risk. A reporter 9not sure if it was on TV or NPR...) asked people in a produce section two questions: "Are you avoiding apples because of the Alar scare?" and "Did you wear seatbelts when you drove to the store?" Most folks answered yes to the first and no to the second.
#11 Darcy - if #16 and #21 don't do it for you, Eliezer at Overcoming Bias has written a sweet explanation of the principle.
Thanks for a most excellent article on one of the ways cognitive bias can screw us. Nick Bostrom, John Leslie, and those guys have written on intuition going wrong in the other direction - tending to unrealistically discount the probability of an extinction/civilization-ending event. Clearly, our intuition is wrong in all directions!
@11 and 21:
- The test result is incorrect 9,999 out of 1,000,000 times.
- IF the test result is positive, then THAT is incorrect 9,999 out of 10,000 times. (This happens in 10,000 out of 1,000,000 tries).
Bush and Blair, the bosom buddies they were, KNEW that they could spread agenda through 9/11 hysteria. People were afraid because of a terrorist attack, it was horrible what happened but as likely as getting hit by a meteor. But people don't care, death is scary and when people die amid a media frenzy, it seems so small. People were so shocked and wanted to have something to feel safe about, we started the DHS and the War in Iraq. Why didn't they start the Peanut Death Prevention Agency (I think someone said it was as likely to die in a terrorist attack as to die from peanuts.) I was in elementary school in 2001 and I remember all the kids that left and said they went to the country. Yeah, right, a school in Texas is a primary target for terrorists. The TERRORists have won.
I leave you with a scene from the Simpsons where flu was going about.
"What do we do?"
"How do we stop the flu!"
"Anything I give you would be a placebo."
"Where do we get these placeboes?"
"They might be in this truck!"
(mass panic)
#18,
You went completely over my head there, I'm afraid.
There's been knife work here, so the numbering has changed. There was an exceptionally puerile troll, so I tossed my non-existent dignity over my shoulder and responded in kind.
Great article, Cory. I think you're 99% correct.
Now write one about the statistical improbability of the US government rounding up liberals and carting them off to detention centers.
@ assumetheposition-
yeah-
I meant to comment on that one- but the ones that got it, got it right. The ones that didn't get it & only made fun of it- well, what's to say to them!
It staggers the mind how anyone can read that article you just referred to, that is linked to there, and not understand what is going on. Heavy doses of denial is all it is.
I wonder how much of who we are -- our goals in life, the kind of friends we make, the causes we fund, even our political affiliations -- is determined by our fears, both rational and irrational. Might make a good Master's thesis for a budding social scientist out there.
I think that basing the anti-war-on-terror argument on the supposed difficulty or rarity of terrorist action is a poor tactic. While, in the US, to date, deaths due to terrorism have been astronomically low-frequency, a cursory examination of potential avenues or tactics for terrorist action and mass murder in the present and near future suggests that it is the exact opposite of difficult - that in fact, mass murder or economic disruption or whatever is horrifically easy.
Take, for example, foot and mouth disease. A relatively recent outbreak in North Carolina spread to 6 other states in
The point is, there is *nothing* we can realistically do to stop this, or events like it. It takes cursory research to come up with the idea, a bit of savvy to get a live sample, and zero effort or risk to implement the actual infection.
This is the power and promise and terror of the age: the power of the individual. To create and produce, by means of new technology, and to cause mass destruction, by the same paths. And there is nothing we can do about it - genie is out of bottle, as it were.
So, the issue is, do we lower the quality and signal-to-noise ratio of our lives and restrict our freedom in order to put on a production of Security Theatre, hoping that it will make us feel better about the awful, inevitable, sublime possibility of annihilation hanging over us, or do we embrace the empowered individual, accept the threat of uncontrollable, unpredictable disaster, and try to live our lives as freely and tolerantly and meaningfully as we can?
In the face of death, we should be impelled to live better and brighter and faster, not to cower in fear and submit our society further to mechanistic, corporatized, intrusive control.
see also: Black Swan Theory
The best line in the piece:
I agree wholeheartedly. As you noted, human beings aren't wired for statistical analysis; it's counter intuitive, and the REALLY useful conditional stuff the general public has hardly any idea about at all. (Makes my spam filter run like gangbusters, though!)
Education isn't a cure-all, but it'll get us a helluva lot further than x-raying everyone's toothbrushes.
Sitting still, shutting up, and accepting the status quo is just what the government wants, and if we do that then the government has already won. Or maybe the terrorists have already won. Errr wait ... who's the enemy supposed to be again?
(And, please, different from, not different than. If people keep saying different than then the Chicago Manual of Style will have to list it as a valid formulation and those of us who prefer the old form will be stuck with the new one for the rest of our lives.)
(I guess that makes me a Fiscal Moderate, a Social Liberal, and a Grammar Conservative.)
This is nothing. I don't know why we didn't go mad during the worst of the Cold War. If you are too young, or have somehow forgotten, we came close to blowing up the whole fucking world in October, 1962! All the revisionist dithering notwithstanding, it was a very REAL possibility that we were blundering into hell. It wasn't forty lunatic Iraqis armed with box cutters, it was thousands of otherwise rational people armed with nuclear weapons who were being carried along by what seemed to be an irreversible momentum. That was REALLY scary. This post-9/11 hysteria is pitiful by comparison. A nation of silly-assed scaredy cats being manipulated by protofascists! Get a grip. It's just terrorism, not terror.
correction: '...lunatic SAUDIS...'
Sixty-two thousand four hundred repetitions make one truth.
It almost worked on even you for a second there.
I couple months back I caught myself almost making the same mistake. I decided from then on that every time I hear someone talking about Iraqi involvment in 9/11 I'm going to say out loud, "there is no connection between Iraq and 9/11". A little self-counter-programming.
it's good to call to mind every now and then that we all have to die of SOMETHING. When exactly did this culture of fear of death creep into American consciousness? The Forties saw the fatalism of real war, the Fifties lived under the mushroom cloud as a real possibility (ask someone - it was NOT abstract), I guess the Sixties was the beginning of the shift. I once read somewhere that the Baby Boomers were the first generation that was raised to think death was optional. Hence the first Youth Worship culture. Is this present incipient police state their legacy? Anything but death? Take my freedom, my money, my peace of mind, my hope for the future, my right to trust other people, my freedom to travel ---- please just don't kill me?
we all have to die of SOMETHING.
That's rich coming from you. The bends? Tentacle rot? Trans-dimensional ague?
http://en.wikipedia.org/wiki/Terror_management
Terror management, literally, as government policy?
what is the cure for fear?
what is the cure for fear?
Ketel One, up, twist.
love that dutch courage - but wrong.
A friend's daughter recently finished a Ph.D in biology and, after the publication of a book on invasive species, is on her way with an academic career. A few years ago she and a friend stayed at my place for a couple of days of beachcombing. One night we watched 'Dr. Strangelove', a movie neither of them had ever seen. When it was finished they were strangely quiet, although they claimed to have loved it. Finally she asked, 'How true was it, the part about the Fail Safe missions?'
She was twenty-one by then, and old enough to know the truth, so I told her that every night of her childhood, while safe in her bed in Seattle, American B-52s flew overhead on such missions, each armed with hydrogen bombs. She was almost mortified that she didn't know this.
'It's not something we told our children,' I said. 'It's the stuff of bad dreams. Childhood's hard enough.'
'My mother, my father ... you and Carole ... your friends ... Wow, no wonder you guys drank so much!'
The yogis say that of the five kleishas, abhinivesha, clinging to life, is the one that can never fully be conquered. Even the most enlightened practitioner, who has completely accepted death, still feels some remnant of fear at the last. Self-preservation is in the cells themselves. I look forward to death as an exciting adventure, but I'm still going to scream if a tiger jumps out at me.
Count coup on a nation with enough bombs to atomize you ten thousand times over?
No, that's been tried, didn't really work out well for anyone.
We all used to play in the neighbor's bomb shelters when we were children.
Yep. Facing your fear works great for irrational fears, but like Antinous said, it's in our cells. The genes of anyone not responding with vigorous avoidance to the idea of their end have long since been asked to leave the pool.
Which leaves ... a river in Egypt? Show me another way if you can, but I've looked and I haven't found it.
Ridley Scott got it exactly right in Bladerunner when he had Rutger Hauer saying "not ... yet!"
Brrrr. Antinous, you've named my oldest nightmare. I have it about once a year: A tiger crouching in my path, lashing its tail.
umphff...*rustle*.... "The Telling of the Hackyneyed Yet True,Episode 13568, please deposit donations in empty sake jar at entrance, thank you honoured guests: (coughcough hyaakkkk-pheuh!) And it came to pass a samurai of ill character did espy an old but comfortably wealthy man in the public square, He then did contrive to entangle scabbards with him in passing. "Ho! ye old clot! What means your insolence! Answer, apologize or fight!" The old man, samurai by dint of service to his daimyo as Tea Master, but no swordsman or warrior, essayed a quavering bow and begged off to the morrow - all witnessing eyes upon him. The evil challenger strode off chuckling,certain of a fat bag of silver coins and grovelling brushed apology on his doorstep by morn.
The Tea Master repaired to his home and then set out again to the Sword Master's. There the Master did hear his tale and pause a moment to reflect. At last he spoke and bade the Tea Master to make Tea. Mystified, since the Tea Master had only come to learn the rudiments of duel to avoid total disgrace in his sure death the next day,he did Make the Tea Ceremony. Narrow-eyed the Sword Master watched. Upon ending, he spoke. "Go ye tomorrow to the appointed place. Compose your mind as for making Tea, raise the sword you wear as Samurai, and when the other closes, strike. It will end in mutual slaying.
The Tea Master bowed his thanks and hurried away. That night he slept like any other. On the dawn, he met the wicked samurai on the death grounds.
There among the crucified remains of arsonists and traitors,he faced his opponent, did prepare the Tea in his mind and raised his sword - and waited.
The false samurai now faced his prey and looked into his eyes. There he found nothing. The Tea Master waited, sword high, life dismissed. The samurai broke. In the face of mutual slaying he sheathed his blade, bowed, begged pardon and slunk away, money not enough to trade life for.
does the cure have to make the fear go away or just have you function correctly even while you feel fear?
If the latter, then courage vow.
Actually, being killed by a large feline is one of my preferred death scenarios. It's quick, it's dramatic and, if a cat's going to eat your face anyway, it's more dignified if it's not an orange tabby.
My wife works at UC Berkeley and they recently had a campus event regarding preparedness and response plans in case of a shooter on campus. This was, no surprise, motivated by the Virginia Tech massacre.
To better understand the issue, she read through most of the 146-page report of the Virginia Tech Review Panel. According to the report, there are on average 16 murders per year across approximately 4,000 university campuses in the US. Before Virginia Tech, the last mass murder on a university campus was when Charles Whitman killed 16 people on the University of Texas campus in ... 1966.
While there have been multiple murders on campuses in the intervening years, none of them involved a murder followed later by another murder by the same killer on campus.
oooh the tiger!
(get the donation jar out!) A man was chased by a tiger in the jungle. Running in his terror, he came to a cliff - the tiger hot behind him. Cornered with no way out he leapt! As he fell he reached blindly out and grasped a vine and narrowly broke his fall,dangling at the cliff edge. As he hung swaying above a deadly drop, the tiger's face appeared over the edge, snuffling and growling. Looking down the killing height he say the tiger's mate emerge from the brush and sit below staring up at him. As he swung slowly back and forth from the vine, two mice, one black and one white, emerged from a hole in the cliff and began nibbling at the vine. The mad espied a wild strawberry growing within reach among the foliage. He reached out and plucked it. How sweet it tasted!
thank you, thank you, I got a million of 'em!
Actually, being killed by a large feline is one of my preferred death scenarios. It's quick, it's dramatic and, if a cat's going to eat your face anyway, it's more dignified if it's not an orange tabby.
Just be sure to say yes in response to I can has cheezburger?
Takuan:
That would be a great concept for a restaurant: Imminent Death -> Food Tastes Great! Lunch under a piano held up by one solitary fraying rope! Desert can be Russian Roulette: 1 bullet is real, the other 5 are jellybeans! And, of course, there's that poison Japanese fish that has to be prepared right or it'll kill you. There's always that.
"Statisticians speak of something called the Paradox of the False Positive. Here's how that works: imagine that you've got a disease that strikes one in a million people, and a test for the disease that's 99% accurate. You administer the test to a million people, and it will be positive for around 10,000 of them – because for every hundred people, it will be wrong once (that's what 99% accurate means). Yet, statistically, we know that there's only one infected person in the entire sample. That means that your "99% accurate" test is wrong 9,999 times out of 10,000! "
This isn't correct. 99% accuracy means that the disease will be detected 99 out of 100 times it actually occurs (accuracy being defined as the correspondence with the "true" condition), but does not say anything about the false-positive rate. Using the example of a million people with one "actual" positive, it means that the diseased person has a 99% chance of being correctly diagnosed by the test and says nothing about the other 999,999. A test that was 99% accurate but gave a 99.99% false positive rate as you describe would never be used, simply because everyone tested would be given a positive diagnosis (except for one in 10,000).
TAKUAN@#49,
Mutually Assured Destruction. I wonder how many people in the Pentagon and Kremlin knew this story?
It's the first time I've heard it. Thanks.
Learning about statistics is good, but you have to learn all about statistics. The article talks about mean, or expected value, but makes no mention of variance.
To begin with the gambling example: casinos make a very small return on the games overall. The advantage to the house in some games is less than 1%. But because they play so many games for such high stakes, that advantage adds up, and even if there is a big winner, the house wins in the long run. The gamblers are not, for the most part, stupid. They know that in the long run, the house wins, but they also know that they are such a small part of the house's business, that variance might hit and they can be a big winner.
And the same thing applies to terrorism. Moving the threat from rare occurrences to daily inconveniences may be a bad deal in the long run, but in the short run, a terrorist attack is much worse. You can plan to deal with the crap at the airport; you can't plan for a terrorist attack.
Which is not to say that the measures taken at the airport make any sense. They don't move the threat of terrorism, they leave it there and add new threats. That's just wrong. But if there is a way to stop terrorism by minor inconvenience, we should think long and hard before rejecting it out of hand. The value in being able to anticipate and plan may be worth it.
%56: It's possible that you're right, but that's because "99% accurate" doesn't tell us which accuracy it is measuring -- the false positive or the false negative.
However, I know from having administered AIDS tests and some familiarity with the subject that "99% accurate" is usually the false-positive measure. The false-negative measure has to be way lower than that to be considered acceptable. The rational is that a false positive is distressing, but causes no actual damage, while a false negative can have massive consequences.
Hazard, compiled by A.G.E. Blake from the unpublished writings and talks of John G. Bennett
http://en.wikipedia.org/wiki/John_G._Bennett
The above is very interesting and goes nicely with early chaos theory. "It is a major step to really accept that there is nothing absolute, fixed or certain..." We do things to try to buffer our minds against this truth. Embracing more security so we could manage some degree of the world's chaos was a rational choice at the time.
Nice article. I would like to add one thought. A major reason why we ignore probability in our daily affairs may be that humans evolved in a society where all news was local and therefore relevant. If caveman Davidwat knew about a bad event, for example the murder of a child, it was because it happened nearby. It implied a local threat that needed attention, in this case a murderer wandering around somewhere close. Modern Davidwat, however, hears about countless bad things that happen all over the world even though they have no actual implications for his current situation. If a child is hurt in Seattle, he unreasonably worries about his son in Dallas. This is a reaction which may have significantly less survival value now that it did 15,000 years ago.
davidwat - Dallas
The numbers bit is completely foolish as presented, specifically because of the lack of differentiation between false positives and false negatives. Plus there is the issue that you only get 9,999/10,000 wrong if you run 1MM samples, so it really is wrong 9,999/1MM (or 1%) of the time, just like it was supposed to be.
Other than that, a great commentary on society and our inability to differentiate between real danger and things that sound good. The real reason people like to legislate and control against the statistically rare event is that they can claim success. It is the same reason the Flying Spaghetti Monster bit worked, though in this case much more on the oppressing civil liberties side and less on the entertainingly engaging in a debate with zealots.
Late to the party, as usual, but from my experience as a System Safety Program Manager, risk consists of more than the statistical probability of an event. Risk is a larger concept that also includes the severity of the event. In the space business, we estimated it as Probability times Severity (R = P * S). So a high-probability event with low severity could be equivalent to a low-probability, high-severity event.
Best,
Gray Rinehart