Thursday, October 14, 2004

Yet more biased sampling

There is a story that during WWII Allied Bomber Command noticed they were losing too many bombers over germany. They set up a team to study the problem. The team requisitioned a collection of squads of soldiers who went all over southern britain looking at shot-down bombers. They recorded every bullet-hole and flak-hole in the shot-up airplanes, and eventually produced a map of the bomber showing where the planes were hit the most. They recommended armor at those spots. The armor would make the planes heavier so they would have shorter range or less payload or less speed, but it was agreed that this price was worth paying if fewer bombers were lost. The recommendations were followed.

Another team kept track of bomber losses. The missions changed, and there were more missions, so it was complicated, but this other team eventually concluded that losses were at least as high as before and maybe higher. The armor was useless or worse than useless. So the first team was assigned to repeat their work and if possible find out what had gone wrong.

While the repeat study was in progress, one of the soldiers they had counting holes in aircraft pointed out to them, "You're only making me count the holes in the planes that made it back." Sure enough, when they looked at the data they found that the spots that were shot up the most often were spots that had no vital function. The most vital spots (like the pilot's seat) were hardly shot up at all. Because those planes didn't make it back across the channel to get measured.


Blogger Joe said...

I like the story. It really reminded me of what I learned in a course last year on aircraft flight mechanics, our prof used to work for the NTSB. She said that most aircraft accidents are blamed on "pilot error," mostly due to a pilot not being able to come back and defend himself after an accident(most are dead). It was almost scary becuase we had a weekly presentation of as she put, "gruesome aircraft mishaps," and almost all of them were blamed on pilot error and as such, any mechanical problems that contributed to the crash do not have to be fixed.

Anyways, I enjoyed your postings on biased sampling. Have a merry Christmas!

9:22 AM  
Blogger elendil said...

It's always easy to know where we go wrong logically in retrospect ... There's a website that does a version of the classic 'pick a card, any card' trick. I think you might like it:

(I'll confess that I cheated and read the answer in an article before I played it.)

7:11 PM  
Blogger mandyprehiem63746768 said...

hey, I just got a free $500.00 Gift Card. you can redeem yours at Abercrombie & Fitch All you have to do to get yours is Click Here to get a $500 free gift card for your backtoschool wardrobe

4:37 AM  

Post a Comment

<< Home