I want to point to an exchange from The Spirit Level authors Richard Wilkinson and Kate Pickett to again illustrate the lack of scientific rigor from these two authors.
The Spirit Level claims that inequality within American states causes lower life expectancy.
When Princeton Professor Angus Deaton and UIUC professor Darren Lubotsky controlled for racial composition, this effect vanished.
University of Massachusetts Amherst scholars Ash and Robinson wrote a reply, saying that they could find an effect, with a certain specification.
Deaton and Lubotsky responded:
“Ash and Robinson… consider alternative weighting schemes and show that in one of our specifications, in one data period, and with one of their alternative weighting schemes, income inequality is estimated to be a risk factor. All of our other specifications, as well as their own preferred specification, replicate our original result, which is supported by the weight of the evidence. Conditional on fraction black, there is no evidence for an effect of income inequality on mortality.”
Angus Deaton is not just any guy. Unlike Wilkinson and Pickett, he is a scientific heavy-weight. When he came to the University of Chicago the last time to speak about econometric methodology, I saw at least 3 Nobel Prize winners in the audience.
Furthermore, as any economist knows, with multiple variables and no clear model, you can data-mine any result you want in 1-2 specifications. If you have to use correlations, what matters is the weight of the evidence.
So how do Wilkinson and Pickett present this exchange to their non-academic , trusting audience?
They simply write that Ash and Robinson find a result, neglecting to even mention the Deaton and Lubotsky study - the one actual researchers take seriously, the one that has 145 citations in google scholar. The Ash and Robinson study in comparison has 1 (sic) citation (although in fairness it is 6 years newer).
If you only read and trust Wilkinson and Pickett, you would not even know the Deaton and Lubotsky study even existed!!!
Wilkinson and Pickett have weak evidence in terms of causation, so they rely a lot on falsely claiming scientific consensus. But if you read the articles, the truth is that the field of research they are engaged in is best described by a consensus that the inequality and health have not been convincingly established to be causally connected, a lack of clean identification, and with the weight of the evidence going against Wilkinson and Pickett.
What academics know, and the broad public does not, is that if you lack a clear model, lack clean experiments, and have multiple variables, have unobservable variables, and when what you are studying can be suspected to be endogenously related (such as health, race and income inequality), you can almost ALWAYS establish a result in some specification, through sufficient data-mining.
In this situation it is very important to rely on the best studies, rely on the weight of the evidence and model averaging, and if you theoretically have reason to belive some varible may be important (such as race), to include the control if possible.
But the scientific standard that Wilkinson and Pickett use is that if someone somewhere finds a correlation in one specification (that they like), that is the end of the story! Proof! Scientific Evidence!
They can ignore all the other research, even if written by much more senior researchers, in more prestigious journals, with better methodology, and go straight to the naïve public and claim that science has proven Wilkinson and Pickett’s ideology true.
Journalists should know this the next time they read Wilkinson and Pickett claim to objectively present scientific evidence.
No comments:
Post a Comment