Statistics and medicine

[haggholm]

During my coffee break, I read an article in Scientific American Mind called Knowing Your Chances (available online). I think it is an outstanding article, and you should read it. The most evocative part may have been a simple example:

Consider a woman who has just received a positive result from a mammogram and asks her doctor: Do I have breast cancer for sure, or what are the chances that I have the disease? In a 2007 continuing education course for gynecologists, Gigerenzer asked 160 of these practitioners to answer that question given the following information about women in the region:

• The probability that a woman has breast cancer (prevalence) is 1 percent.
• If a woman has breast cancer, the probability that she tests positive (sensitivity) is 90 percent.
• If a woman does not have breast cancer, the probability that she nonetheless tests positive (false-positive rate) is 9 percent.

What is the best answer to the patient’s query?

1. The probability that she has breast cancer is about 81 percent.
2. Out of 10 women with a positive mammogram, about nine have breast cancer.
3. Out of 10 women with a positive mammogram, about one has breast cancer.
4. The probability that she has breast cancer is about 1 percent.

Before you read on, take a brief moment to think about it, but also note your gut feeling. Done? Let’s continue:

Gynecologists could derive the answer from the statistics above, or they could simply recall what they should have known anyhow. In either case, the best answer is C; only about one out of every 10 women who test positive in screening actually has breast cancer. The other nine are falsely alarmed. Prior to training, most (60 percent) of the gynecologists answered 90 percent or 81 percent, thus grossly overestimating the probability of cancer. Only 21 percent of physicians picked the best answer—one out of 10.

Doctors would more easily be able to derive the correct probabilities if the statistics surrounding the test were presented as natural frequencies. For example:

• Ten out of every 1,000 women have breast ­cancer.
• Of these 10 women with breast cancer, nine test positive.
• Of the 990 women without cancer, about 89 nonetheless test positive.

Thus, 98 women test positive, but only nine of those actually have the disease. After learning to translate conditional probabilities into natural frequencies, 87 percent of the gynecologists understood that one in 10 is the best answer.

I’m happy to say that I did get it right on the first try, but I strongly agree witht the authors’ opinion that it is not intuitive when the statistics are cited as probabilities rather than natural frequencies. The reason I got it right is because I’ve done a bit of math and a wee bit of stats, I enjoy reading some blogs that talk about medical statistics, I know some of the not-quite-obvious ground rules of probabilities; I know what Type I and Type II errors are (even if I occasionally mix them up)…

…And, perhaps crucially, I’ve spent time thinking about false positives in medical testing before. When I get my periodic routine screenings for STIs (I’ve never had symptoms or tested positive for any, I’m glad to say, but I feel a responsible person should get tested anyway!), I’ve asked myself the hypothetical question What if it did show positive for, say, HIV? What are the odds that I would actually have it? (It turns out that if you’re a heterosexual male, and if you test positive for HIV, there’s about a 50% chance that you don’t have it! You should play it safe, but get re-tested and don’t panic. Some people commit suicide when they get positive test results, even though they’re as likely as not to be healthy.)

Still, while my gut told me the answer was not A (wherein I did better than most of the gynecologists), I had to think about it for a minute to figure out which was the proper answer. People need to be educated on this stuff. Meanwhile, if you haven’t had the benefit of statistical education, keep this one thing in mind: The obvious answer is not always correct, so if you’re unsure, ask someone who can do the maths. And, sadly, even your doctor may not know. I actually find it rather sad that as after learning to translate conditional probabilities into natural frequencies, 87 percent of the gynecologists understood that one in 10 is the best answer, this means that even after simplification, more than 1 in 10 gynecologists didn’t get it. Your doctor can spot the symptoms and order the right tests, but you may need a mathematically inclined friend to actually calculate the risks.