Stegenga on medical nihilism
On EconTalk (a), the whole interview is great.
About the troubles with anecdotal reports of a treatment's efficacy:
Stegenga: Now, what about first-person reports? What about first-person anecdotes? Like, "This drug worked for me." Or, "This drug worked for a good friend of mine," or "a patient of mine?"
...
The short answer is we should approach first-person reports with a huge amount of cautionary skepticism. And this is for three fundamental reasons, that all work together.
The first reason is that diseases have a natural course of progression. That is, they have a kind of a life of their own. So, symptoms get better and worse over time for many diseases. Some diseases have a natural course of progression in which the symptoms gradually decrease until they are gone. This is, for instance, like illustrated by common cold. Some diseases fluctuate with symptoms varying over time. So, for instance, like bipolar disorder, or depression – symptoms are worse at some times, better at other times. And, people tend to seek treatment from their physician when their symptoms are especially bad. Now, if you seek treatment when your symptoms are especially bad, then merely the passage of time alone entails that your symptoms will get better in the future – for these diseases that have a fluctuating severity of symptoms or a gradually decreasing severity of symptoms. So that's problem number one: the natural course of disease.
Problem number two is the infamous placebo effect. So, the placebo effect involves the expectation is that you'll get better because you received treatment from a health care professional, in fact causes you to get better: but not via the biochemical activity of the drug that you've consumed, but via some sort of mysterious psychological phenomenon that we don't actually understand very well at this point. So, that's problem number two: placebo effect.
Problem number three is a well-known fallacy of reasoning that philosophers call confirmation bias.
...
So these three problems together – the natural course of diseases, the placebo effect, and confirmation bias – entail that we should treat first-person reports regarding the effects of interventions with a huge amount of skepticism.
Now, I should add the following caveat, though. In medicine, there's been a long tradition of neglecting the patient's reports, because medicine, at least sometimes, has been kind of imperialistic in its attitudes. So, "the physician is the educated one; they know about your disease; you don't know anything about your disease." You are sick. Maybe you're a woman. Maybe you're disabled... the white, upper middle-class, male physician knows best.
And so there's been a tendency to push back against this in medicine. "Medicine should listen more, should hear the patient, and should respect what the patient is reporting."
I agree with all of that... physicians should be listening very carefully to our patients and respecting what patients report. However, when it comes to causal inference, that's a completely different ballgame. And I think we ought to be maintaining really, really strict evidential standards when it comes to deciding: Did this drug have the following effect?