Petter Häggholm’s blog

PSA: In spite of a recent press release from an organisation no less respectable than the WHO, and the consequent media furor from less respectable sources, there is in fact no good reason to think that cell phones are responsible for increases in cancer. A few facts:

• The WHO has reported on the same research before. Last time, they interpreted it as meaning that cell phones don’t increase risk of cancer. This time, they seem to interpret it as meaning that they do. As far as anyone knows, nothing’s changed except their reporting.

• There have been many studies on this, the general concensus of which are that there’s no link between cell phone use and incidence of brain cancers. In fact, it’s not even plausible on a basic science level.

• The “smoking gun” study has an enormous flaw: It’s a retrospective study based on asking brain cancer patients about their cell phone usage a decade ago. Would it be surprising, given their circumstances, and given that they know that the possibility of causation has been floated about, if they accidentally overestimated their usage?

• Even after what looks to be jumping to conclusions, cell phone usage has been placed in the same rather tentative risk category of cancer causation as—pay attention—pickled vegetables and coffee.

More here (good and succinct and fairly brief), and here (comprehensive and not at all brief).

These days scientists have a much clearer picture of our inner ecosystem. We know now that there are a hundred trillion microbes in a human body. You carry more microbes in you this moment than all the people who ever lived. Those microbes are growing all the time. So try to imagine for a moment producing an elephant’s worth of microbes. I know it’s difficult, but the fact is that actually in your lifetime you will produce five elephants of microbes. You are basically a microbe factory.

The microbes in your body at this moment outnumber your cells by ten to one. And they come in a huge diversity of species—somewhere in the thousands, although no one has a precise count yet. By some estimates there are twenty million microbial genes in your body: about a thousand times more than the 20,000 protein-coding genes in the human genome. So the Human Genome Project was, at best, a nice start. If we really want to understand all the genes in the human body, we have a long way to go.

[…]

In the September 2010 issue of the journal Microbiology and Molecular Biology Reviews, a team of researchers looked over this sort of research and issued a call to doctors to rethink how they treat their patients. One of the section titles sums up their manifesto: “War No More: Human Medicine in the Age of Ecology.” The authors urge doctors to think like ecologists, and to treat their patients like ecosystems.

[…]

Here’s one crude but effective example of what this kind of ecosystem engineering might look like. A couple years ago, Alexander Khoruts, a gastroenterologist at the University of Minnesota, found himself in a grim dilemma. He was treating a patient who had developed a runaway infection of Clostridium difficile in her gut. She was having diarrhea every 15 minutes and had lost sixty pounds, but Khoruts couldn’t stop the infection with antibiotics. So he performed a stool transplant, using a small sample from the woman’s husband. Just two days after the transplant, the woman had her first solid bowel movement in six months. She has been healthy ever since.

Carl Zimmer, The Human Lake

As I was re-reading Jerry Coyne’s Why Evolution Is True the other night, an aside about bacterial resistance to antibiotics set me to thinking. It’s well known, and to be expected from evolutionary theory, that bacteria—and other pathogenic organisms, such as viruses and eukaryotic parasites—tend to evolve resistance to the drugs designed to wipe them out. It’s fairly obvious: Those drugs exert an extreme selection pressure, as strong as we can make it…thus if the variation exists to produce resistance, we’re positively selecting for it and are guaranteed to end up with it in the end.

As Coyne also pointed out, the response is pretty variable: Diseases like polio and measles haven’t seemed to evolve any resistance at all, while common flu strains evolve so quickly that every flu season is a gamble, where the vaccines available are but a best guess as to what will crop up next. HIV takes the prize as the grand champion of rapid evolution: It evolves so quickly inside each carrier that it eventually evolves its way around the immune system.

This of course reminded me of Orgel’s Second Rule: Evolution is cleverer than you are. As long as medical scientists invent drugs to eradicate pathogenic organisms, they will be exerting selective pressures and will tend to select for resistance. I don’t see that there’s a feasible way around this—short of a drug so astonishingly good that it is always 100% effective (even in the face of imperfect patient compliance!), or a set of rotating drugs so large that no strain can evolve resistance to all at once, it seems impossible; and those conditions seem extremely implausible.

So I wonder, how much medicine is being done that is not only aware of evolution (and the dangers it causes in adaptation to resist drugs), but actually uses it? What follows is very speculative—I’m no biologist or medical expert; I’m just thinking out loud, wondering how much of this is done or has been done; and if not, then why.

One example of a strategy that harnesses evolution would be to design phages. A phage (or bacteriophage), if you don’t know, is a virus that infects a bacterium. This may sound like a rare and exotic sort of being, but phages may well be one of the largest groups of organisms on Earth. Phage therapy makes a sort of intuitive and beautiful sense: Instead of giving a patient a drug that may operate on the basis of differential toxicity (it will damage the bacteria a lot more than the patient), give them something that is absolutely specific to the infectuous agent. (In fact, if Wikipedia is to be believed on this, phage therapy is so specific that this is actually one of the problems: Each phage targets a very specific strain of bacteria, so if the infectuous strain is different…)

But it seems to me that there’s room for so much more. Now that we stand at the early dawn of the age of genetic engineering, perhaps a time will come when phages can be designed to target pathogens for which no useful phages are known. And if the phages we have access to are less than ideal, then why should we not selectively breed them? If we have a disease for which all of Koch’s postulates hold, for instance, and we can retrieve the real, infectuous strains and grow them in culture, then surely we can use those cultures as growth media for phages—evolving phages better able to attack those strains. By wiping out the bacterial cultures and growing subsequent generations of phages on bacteria that lack exposure, the bacteria will have no opportunity to evolve resistance, and we should be able to essentially breed phages to wipe out whatever bacteria we like.

Of course, eventually resistant strains would spread throughout the population. But so what? It’s no big deal. We’d just isolate those strains and selectively breed our phages to target them.

I don’t know whether anything similar could work against viruses. Viruses don’t metabolise, so virophages are pretty damned rare (I’ve only ever heard of one). Are there bacteria that eat viruses? If so, those bacteria could presumably be selectively “bred”…

Another, more modest, evolution-aware strategy would be to deliberately select against virulence. Consider, for example, a hypothetical drug that targets some, but only some strains of flu. Specifically, it suppresses particularly dangerous strains, but is deliberately designed not to target the less harmful ones. Such a strategy would not wipe out flu, of course—but then, we couldn’t do that in the first place. What it might do is give more “benign” strains of flu a competitive advantage over their deadlier cousins. Since all strains of flu work in similar ways, they could be viewed as competitors for the same ecological niche. (I could be wrong about this; I’m speculating, after all!) If we allow a relatively benign strain to fill this niche, but do our best to keep the deadlier version down, then we basically co-operate with the more benign flu at the expense of the more dangerous one. There will be a selective benefit in being benign.

(The inspiration for this speculation is actually evolution in HIV. HIV tends to spread pretty slowly—its epidemic proportions may not make it seem that way, but you really can’t spread HIV as quickly as you can spread a cold. A sneeze just hits more people than—Let’s just say that airborne diseases can spread more quickly. As a result, a strain of HIV that caused the carrier to drop dead within the week would never get off the ground; it would spread more slowly than it killed, which limits a disease to a small group of carriers. In other words, a disease has to spread slowly enough to infect a bunch of other carriers, and how slowly is enough depends on the disease: In case of HIV, pretty slowly. I have read, though I cannot now find a source for this claim, that some widespread strains of HIV are evolving toward slower development to full-blown AIDS, for these reasons.)

I harbour no illusion that any of this is revolutionary to anyone at all professionally competent in biology or medical science—I am at best a reasonably well-read layman. But the notion of evolutionary strategies in medicine are intriguing to me, and I really do wonder how much is being done, how much has been done, how much can and will be done with it. For now, I merely jot down these thoughts and speculations. When I have a little more time, I should do some digging and some reading to see what’s out there.

Doing my part to echo reason in the skeptical blogosphere, I’ll make a brief mention of what I’ve read about the new USPSTF guidelines, which you may have heard of. If not, Dr. David Gorski explains and deconstructs. The short version is, a group belonging to (but not setting policy for) the US government has altered its recommendations for mammographic screening to

1. not screen women aged 40–49 anymore (rather, wait until 50)
2. screen once every two years, instead of annually

Naturally, a lot of people misunderstand this and some of the less reasonable among them start crying about misogynism and the Obama administration’s death panels. These people miss a lot of obvious points.

• These are screening guidelines. They have nothing to do with recommendations for symptomatic women (examining them is not screening, but diagnosis). It also does not apply to women with known risk factors. It’s a change in how they suggest screening of asymptomatic women should happen.

• The group was not set up by the Obama administration. The US authorities do not condone these guidelines. In fact, most groups do not, though if Dr. Gorski is right, a gradual shift in this general direction may happen over time. Either way, the USPSTF just makes recommendations; they have no power to dictate policy.

• The tricky one: The guidelines actually make a lot of sense. Excessive screening does more harm than good.

It may sound bizarre that more cancer screening could be harmful, but it’s true. Apart from discomfort and angst caused by false positive diagnoses, there’s very real pain and even small danger in performing biopsies on harmless lumps (even good medical interventions are never completely risk free). And, not all cancers will kill you—a few may spontaneously go away, but much more significantly, a lot of cancers are just so slow-growing that they shouldn’t be on your list of worries. With the average life expectancy around 80, a tumour that will absolutely kill you by your 110th birthday is…really nothing to worry about. You’re more likely to live longer without the harsh regimen of surgery, chemotherapy, and radiation necessary to treat the cancer, even though that same regimen is an absolute life-saver if you have the sort of cancer that would kill you before you’re certainly dead by natural causes.

There are also other factors, such as lead time bias (highly recommended reading). It’s easy to say that If we screen 40-year-olds, most diagnosed cancer patients survive on average 15 years; if we only screen 50-year-olds, we find that our average patient only survives 5 years and think that early screening lets people live longer (15 years versus 5!)…but I’ve just described two scenarios with people dying at age 55; the difference is how long they live with the knowledge that they have cancer. This sort of thing happens, it is significant, and it confounds trials and policy making. Detecting cancers earlier is only helpful if interventions actually turn out to save lives.

I won’t say much further, because this is obviously not my area of expertise, but because a moral panic has sprung up around the internet, I figured I would say something in case you stumble across my blog. If these issues concern you, I highly recommend David Gorski’s write-up, the SkepChick counter to a bad Feministing report, and Orac’s direct deconstruction of the canards and conspiracy theories.

The US National Institute of Health department, the National Centre for Complementary and Alternative Medicine (NC-CAM), whose aim is to find evidence for alternative medicine, found to its chagrin that alternative medicine doesn’t work. Key snippets:

Ten years ago the government set out to test herbal and other alternative health remedies to find the ones that work. After spending $2.5 billion, the disappointing answer seems to be that almost none of them do.

Echinacea for colds. Ginkgo biloba for memory. Glucosamine and chondroitin for arthritis. Black cohosh for menopausal hot flashes. Saw palmetto for prostate problems. Shark cartilage for cancer. All proved no better than dummy pills in big studies funded by the National Center for Complementary and Alternative Medicine. The lone exception: ginger capsules may help chemotherapy nausea.

As for therapies, acupuncture has been shown to help certain conditions [though if I read it aright, That finding was called into question when a later, larger study found that sham treatment worked just as well –ed.], and yoga, massage, meditation and other relaxation methods may relieve symptoms like pain, anxiety and fatigue.

…

…Critics say that unlike private companies that face bottom-line pressure to abandon a drug that flops, the federal center is reluctant to admit a supplement may lack merit — despite a strategic plan pledging not to equivocate in the face of negative findings.

…

"There's been a deliberate policy of never saying something doesn't work. It's as though you can only speak in one direction," and say a different version or dose might give different results, said Dr. Stephen Barrett, a retired physician who runs Quackwatch, a web site on medical scams.

…

Critics also say the federal center's research agenda is shaped by an advisory board loaded with alternative medicine practitioners. They account for at least nine of the board's 18 members, as required by its government charter. Many studies they approve for funding are done by alternative therapy providers; grants have gone to board members, too.

…

[The Centre’s methodology] is opposite how other National Institutes of Health agencies work, where scientific evidence or at least plausibility is required to justify studies, and treatments go into wide use after there is evidence they work — not before.

…

In a federally funded pilot study, 30 dieters who were taught acupressure regained only half a pound six months later, compared with over three pounds for a comparison group of 30 others. However, the study widely missed a key scientific standard for showing that results were not a statistical fluke.

In other words, NC-CAM, which was founded with the intent of finding evidence for the quackery that the sponsoring Senators were already convinced by (to look for a yes, in other words, rather than objectively assessing credibility), is perfectly happy to spend millions upon millions of US tax dollars on investigating ludicrous fantasies like distance faith healing, energy healing, and homeopathy (dollars that could be spent on valid research), is biased by a board of proponents, tends to publish lackluster studies with missing controls…and still can’t come up with a single positive result beyond noting that ginger may (may) help with nausea.

If that’s the best they can come up with the cards stacked unreasonably in their favour, then it’s time to pull the plug and spend the next $2.5 billion dollars on something useful.

During my coffee break, I read an article in Scientific American Mind called Knowing Your Chances (available online). I think it is an outstanding article, and you should read it. The most evocative part may have been a simple example:

Consider a woman who has just received a positive result from a mammogram and asks her doctor: Do I have breast cancer for sure, or what are the chances that I have the disease? In a 2007 continuing education course for gynecologists, Gigerenzer asked 160 of these practitioners to answer that question given the following information about women in the region:

• The probability that a woman has breast cancer (prevalence) is 1 percent.
• If a woman has breast cancer, the probability that she tests positive (sensitivity) is 90 percent.
• If a woman does not have breast cancer, the probability that she nonetheless tests positive (false-positive rate) is 9 percent.

What is the best answer to the patient’s query?

1. The probability that she has breast cancer is about 81 percent.
2. Out of 10 women with a positive mammogram, about nine have breast cancer.
3. Out of 10 women with a positive mammogram, about one has breast cancer.
4. The probability that she has breast cancer is about 1 percent.

Before you read on, take a brief moment to think about it, but also note your gut feeling. Done? Let’s continue:

Gynecologists could derive the answer from the statistics above, or they could simply recall what they should have known anyhow. In either case, the best answer is C; only about one out of every 10 women who test positive in screening actually has breast cancer. The other nine are falsely alarmed. Prior to training, most (60 percent) of the gynecologists answered 90 percent or 81 percent, thus grossly overestimating the probability of cancer. Only 21 percent of physicians picked the best answer—one out of 10.

Doctors would more easily be able to derive the correct probabilities if the statistics surrounding the test were presented as natural frequencies. For example:

• Ten out of every 1,000 women have breast ­cancer.
• Of these 10 women with breast cancer, nine test positive.
• Of the 990 women without cancer, about 89 nonetheless test positive.

Thus, 98 women test positive, but only nine of those actually have the disease. After learning to translate conditional probabilities into natural frequencies, 87 percent of the gynecologists understood that one in 10 is the best answer.

I’m happy to say that I did get it right on the first try, but I strongly agree witht the authors’ opinion that it is not intuitive when the statistics are cited as probabilities rather than natural frequencies. The reason I got it right is because I’ve done a bit of math and a wee bit of stats, I enjoy reading some blogs that talk about medical statistics, I know some of the not-quite-obvious ground rules of probabilities; I know what Type I and Type II errors are (even if I occasionally mix them up)…

…And, perhaps crucially, I’ve spent time thinking about false positives in medical testing before. When I get my periodic routine screenings for STIs (I’ve never had symptoms or tested positive for any, I’m glad to say, but I feel a responsible person should get tested anyway!), I’ve asked myself the hypothetical question What if it did show positive for, say, HIV? What are the odds that I would actually have it? (It turns out that if you’re a heterosexual male, and if you test positive for HIV, there’s about a 50% chance that you don’t have it! You should play it safe, but get re-tested and don’t panic. Some people commit suicide when they get positive test results, even though they’re as likely as not to be healthy.)

Still, while my gut told me the answer was not A (wherein I did better than most of the gynecologists), I had to think about it for a minute to figure out which was the proper answer. People need to be educated on this stuff. Meanwhile, if you haven’t had the benefit of statistical education, keep this one thing in mind: The obvious answer is not always correct, so if you’re unsure, ask someone who can do the maths. And, sadly, even your doctor may not know. I actually find it rather sad that as after learning to translate conditional probabilities into natural frequencies, 87 percent of the gynecologists understood that one in 10 is the best answer, this means that even after simplification, more than 1 in 10 gynecologists didn’t get it. Your doctor can spot the symptoms and order the right tests, but you may need a mathematically inclined friend to actually calculate the risks.

“Orac” over at Respectful Insolence has a writing style that’s fairly prone to offend—definitely pugnacious, and very fond of side swipes at those he dislikes (primarily alternative medicine quacks)—and I don’t blame him for his distaste, which in fact I share, but it does sometimes make his essays a bit harder to slog through. (He also has an inordinate fondness for beginning sentences with Indeed. This is one area where I can tentatively claim superiority: I can also be pugnacious and come off as offensive, but while I am no less prone than Orac to complicated sentence structure, I’ve never been accused of any such repetitive verbal tic.)

However, those foibles aside, he has written some very good stuff (he’s on my list of blogs I ready daily for a reason), and this article, summarising and explaining the work of a John Ioannidis, was very interesting indeed. The claim it looks at is a very interesting and puzzling one: Given a set of published clinical studies reporting positive outcomes, all with a confidence interval of 95%, we should expect more than 5% to give wrong results; and, furthermore, studies of phenomena with low prior probability are more likely to give false positives than studies where the prior probabilities are high. He has often cited this result as a reason why we should be even more skeptical of trials of quackery like homeopathy than the confidence intervals and study powers suggest, but I have to confess I never quite understood it.

I would suggest that you go read the article (or this take, referenced therein), but at the risk of being silly in summarising what is essentially a summary to begin with…here’s the issue, along with some prefatory matter for the non-statisticians:

A Type I error is a false positive: We seem to see an effect where there is no effect, simply due to random chance. This sort of thing does happen. Knowing how dice work, I may hypothesise that if you throw a pair of dice, you are not likely to throw two sixes, but one time out of every 36 (¹/₆×¹/₆), you will. I can confidently predict that you won’t roll double sixes twice in a row, but about one time in 1,296, you will. Any time we perform any experiment, we may get this sort of effect, so a statistical test, such as a medical trial, has a confidence level, where a confidence level of 95% means there’s a 5% chance of a Type I error.

There’s also a Type II error, or false negative, where the hypothesis is true but the results just aren’t borne out on this occasion. To the best of my knowledge, there is no equivalent of the confidence level for Type II errors.

This latter observation is a bit problematic, and leads into what Ioannidis observed:

Suppose there are 1000 possible hypotheses to be tested. There are an infinite number of false hypotheses about the world and only a finite number of true hypotheses so we should expect that most hypotheses are false. Let us assume that of every 1000 hypotheses 200 are true and 800 false.

It is inevitable in a statistical study that some false hypotheses are accepted as true. In fact, standard statistical practice [i.e. using a confidence level of 95%] guarantees that at least 5% of false hypotheses are accepted as true. Thus, out of the 800 false hypotheses 40 will be accepted as "true," i.e. statistically significant.

It is also inevitable in a statistical study that we will fail to accept some true hypotheses (Yes, I do know that a proper statistician would say "fail to reject the null when the null is in fact false," but that is ugly). It's hard to say what the probability is of not finding evidence for a true hypothesis because it depends on a variety of factors such as the sample size but let's say that of every 200 true hypotheses we will correctly identify 120 or 60%. Putting this together we find that of every 160 (120+40) hypotheses for which there is statistically significant evidence only 120 will in fact be true or a rate of 75% true.

Did you see that magic? Our confidence interval was 95%, no statistics were abused, no mistakes were made (beyond the ones falling into that 5% gap, which we accounted for), and yet we were only 75% correct.

The root of the problem is, of course, the ubiquitous problem of publication bias: Researchers like to publish, and people like to read so journals like to print, positive outcome studies rather than negative ones, because a journal detailing a long list of ideas that turned out to be wrong isn’t very exciting. The problem is, obviously, that published studies are therefore biased in favour of positive outcomes. (If not, all 800 studies of false hypotheses would have been published and the problem would disappear.)

Definition time again: A prior probability is essentially a plausibility measure before we run an experiment. Plausibility sounds very vague and subjective, but can be pretty concrete. If I know that it rains on (say) 50% of all winter days in Vancouver, I can get up in the morning and assign a prior probability of 50% to the hypothesis that it’s raining. (I can then run experiments, e.g. by looking out a window, and modify my assessment based on new evidence to come up with a posterior probability.)

Now we can go on to look at why Orac is so fond of holding hypotheses with low prior probabilities to higher standards. It’s pretty simple, really: Recall that the reason why we ended up with so many false positives above—the reason why false positives were such a large proportion of the published results—is because there were more false hypotheses than true hypotheses. The more conservative we are in generating hypotheses, the less outrageous we make them, the more likely we are to be correct, and the fewer false hypotheses we will have (in relation to true hypotheses). Put slightly differently, we’re more likely to be right in medical diagnoses if we go by current evidence and practice than if we make wild guesses.

Now we see that modalities with very low prior probability, such as ones with no plausible mechanism, should be regarded as more suspect. Recall that above, we started out with 800 false hypotheses (out of 1000 total hypotheses), ended up accepting 5% = 40 of them, and that

It's hard to say what the probability is of not finding evidence for a true hypothesis because it depends on a variety of factors such as the sample size but let's say that of every 200 true hypotheses we will correctly identify 120 or 60%. Putting this together we find that of every 160 (120+40) hypotheses for which there is statistically significant evidence only 120 will in fact be true or a rate of 75% true.

That is, the proportion of true hypotheses to false hypotheses affects the accuracy of our answer. This is very easy to see—let’s suppose that only half of the hypotheses were false; now we accept 5% of 500, that is 25 false studies, and keeping the same proportions,

…Let's say that of every 200 500 true hypotheses we will correctly identify 120 300 or 60%. Putting this together we find that of every 160 (120+40) 325 (300+25) hypotheses for which there is statistically significant evidence only 120 300 will in fact be true or a rate of 75% 92% true.

We’re still short of that 95% measure, but we’re way better than the original 75%, simply by making more plausible guesses (within each study, we were still equally likely to make either Type I or Type II errors). The less plausible an idea is, the higher the proportion of false hypotheses will be out of all the hypotheses the idea generates: A true/false ratio. Wild or vague ideas (homeopathy, reiki, …) are very likely to generate false hypotheses along with any true ones they might conceivably generate. More conventional ideas will tend to generate a higher proportion of true hypotheses—if we know from long experience that Aspirin relieves pain, it’s very likely that a similar drug does likewise.

This is not to say that no wild ideas are ever right. Of course they sometimes are (though of course they usually aren’t). What it does mean is that not only should we be skeptical and demand evidence for them, there are sound statistical reasons to set the bar of evidence even higher for implausible than for plausible modalities.

It is also a good argument for the move away from strict EBM (evidence-based medicine) to SBM (science-based medicine) where things like prior probability are taken into account. Accepting 95% double-blind trials at face value isn’t good enough.