False Positives

A recent article in the journal Demography (November 2005) discusses, in an interesting way, the relationship between longetivity (how long you live) and non-morbid obesity (body mass index over 25 but less than 40, the threshold for morbid obesity). Instead of focusing on obesity of individuals and how it affects their mortality outcomes, Gronniger, a researcher at the US Congressional Budget Office (don't ask me why) looks at whether an individual has a family member who is obese (he calls this familial obesity) and see how it influences the individual's own mortality outcome. Yes, it sounds implausible that someone else's obesity should adversely affect your health, at least we would not think there is a direct effect.

But that is precisely the idea. If the risk profile coming from estimation using individual obesity measure was to be intrepreted as the 'correct' profile of how obesity influences your longetivity, replacing the measure with familial obesity should give a different risk profile, one with less or maybe even negligible effects of (familial) obesity. Lo and behold, not only was familial obesity statistically significant, the risk profile from using this measure turned out to be similar to the one using individual obesity measure. In other words, the adverse health effect of having someone who is obese in your household is similar to actually being obese yourself. Gronniger argued that this suggests that the estimates using individual obesity measure are actually capturing a lot of household characteristics that are correlated with, but being unobserved, falsely attribruted to, obesity. Familial obesity effectively acts as a proxy for these unobservables. He then concludes that many previous studies that ignored these unobservables may have overestimated the mortality risks of non-morbid obesity.

This approach, identifying false positives, are not unique, we do see it from time to time. Indeed, the paper refers to a well-known study by DiNardo and Pischke (Quarterly Journal of Economics 1997) who studied the effects of various office equipments on wage and productivity. Their study was motivated by a previous study by Krueger (QJE 1993) who argues that the use of personal computers has change the wage structure in the US, after finding that controlling for workers characteristics, those who use personal computers earn 15 to 20 percent more than those who don't. DiNardo and Pischke argue that the observed differential was more likely a reflection of the difference between the types of worker (i.e., selection problem, by now a standard objection) but they show it in a clever way. They use a micro data from Germany and run their estimation using data on personal computers use and also found a wage differential, just like in the US study. Then they performed similar estimation using calculators, telephones, pens, and pencils, and found similarly-sized wage differentials! Their paper was cheekily titled "The Returns to Computer Use Revisited: Have Pencils Changed the Wage Structure Too?"

The examples above show us what those of you who have been frequenting this Cafe already know: we are a skeptical bunch when it comes to claims about causality (this, I believe, is also one of Roby's pet peeves). In the absence of a truly random experiment, it is difficult in social science to show empirically the relationship between one variable and another without running into all sorts of statistical problems, even more so to establish causality. And I think this is not some technical details that only those in research need to worry about. In recent policy debates, we hear policy makers, scholars, and pundits make a lot of sweeping conclusions, a number of which ought to be put under more scrutiny, for examples: "the increase in gas price causes the increase in the number of malnourished kids", "the increased availability of porn causes an increase in rape cases in Indonesia", and so on. Just listen to what the social issue du jour is and you'll hear similar statements. I'm not saying they're false. Most of the times we don't know the answers. Yet. Finding false positives is not the only way to answer these questions, but it may direct us closer to the truth. Like they say, a little dose of skepticism is always healthy.


Empirical