Thursday, February 23, 2006

False Positives

A recent article in the journal Demography (November 2005) discusses, in an interesting way, the relationship between longetivity (how long you live) and non-morbid obesity (body mass index over 25 but less than 40, the threshold for morbid obesity). Instead of focusing on obesity of individuals and how it affects their mortality outcomes, Gronniger, a researcher at the US Congressional Budget Office (don't ask me why) looks at whether an individual has a family member who is obese (he calls this familial obesity) and see how it influences the individual's own mortality outcome. Yes, it sounds implausible that someone else's obesity should adversely affect your health, at least we would not think there is a direct effect.

But that is precisely the idea. If the risk profile coming from estimation using individual obesity measure was to be intrepreted as the 'correct' profile of how obesity influences your longetivity, replacing the measure with familial obesity should give a different risk profile, one with less or maybe even negligible effects of (familial) obesity. Lo and behold, not only was familial obesity statistically significant, the risk profile from using this measure turned out to be similar to the one using individual obesity measure. In other words, the adverse health effect of having someone who is obese in your household is similar to actually being obese yourself. Gronniger argued that this suggests that the estimates using individual obesity measure are actually capturing a lot of household characteristics that are correlated with, but being unobserved, falsely attribruted to, obesity. Familial obesity effectively acts as a proxy for these unobservables. He then concludes that many previous studies that ignored these unobservables may have overestimated the mortality risks of non-morbid obesity.

This approach, identifying false positives, are not unique, we do see it from time to time. Indeed, the paper refers to a well-known study by DiNardo and Pischke (Quarterly Journal of Economics 1997) who studied the effects of various office equipments on wage and productivity. Their study was motivated by a previous study by Krueger (QJE 1993) who argues that the use of personal computers has change the wage structure in the US, after finding that controlling for workers characteristics, those who use personal computers earn 15 to 20 percent more than those who don't. DiNardo and Pischke argue that the observed differential was more likely a reflection of the difference between the types of worker (i.e., selection problem, by now a standard objection) but they show it in a clever way. They use a micro data from Germany and run their estimation using data on personal computers use and also found a wage differential, just like in the US study. Then they performed similar estimation using calculators, telephones, pens, and pencils, and found similarly-sized wage differentials! Their paper was cheekily titled "The Returns to Computer Use Revisited: Have Pencils Changed the Wage Structure Too?"

The examples above show us what those of you who have been frequenting this Cafe already know: we are a skeptical bunch when it comes to claims about causality (this, I believe, is also one of Roby's pet peeves). In the absence of a truly random experiment, it is difficult in social science to show empirically the relationship between one variable and another without running into all sorts of statistical problems, even more so to establish causality. And I think this is not some technical details that only those in research need to worry about. In recent policy debates, we hear policy makers, scholars, and pundits make a lot of sweeping conclusions, a number of which ought to be put under more scrutiny, for examples: "the increase in gas price causes the increase in the number of malnourished kids", "the increased availability of porn causes an increase in rape cases in Indonesia", and so on. Just listen to what the social issue du jour is and you'll hear similar statements. I'm not saying they're false. Most of the times we don't know the answers. Yet. Finding false positives is not the only way to answer these questions, but it may direct us closer to the truth. Like they say, a little dose of skepticism is always healthy.


2 comments:

  1. komentar umum: pak ujang, menyambung obrolan kita di mobil waktu beli pizza soal 'perang ideologi' di indonesia. mungkin ada bagusnya posting ini dan cafe salemba secara umum pake bhs indonesia. karena kalo pake bhs inggris yang baca kebanyakan yg udah seideologi. padahal 'lawan kita' yang perlu di 'convert' mungkin lebih banyak hanya berbahasa indonesia. :D tapi pasti ada pertimbangan lain.

    ReplyDelete
  2. Roby, thanks for the comment/advice. Before the rest of you are up in arms about the "ideology war" mentioned by Roby, let me make it clear: it's not about neoliberalism vs ekonomi kerakyatan, etc. It's more about how hard it is to convince people to appreciate (not worship) the quantitative approach and positivism in social science that the resistance often seems 'ideological'. There.

    We appreciate the concern that by posting in English, the Cafe may just be preaching the choir, prosetylizing the converted. We prefer and will continue to post in English for a number of reasons, satisfying a captive audience is not one of them.

    My lazy answer is that we find ourselves in an equilibrium where the writings and comments are in English and it's going to take a strong exogenous shock or perturbation to move us from this equilibrium (what is it with economists and their equlibria). Besides we do have some postings in bahasa Indonesia from time to time. All right, we did it once .

    Anyway, we have a little poll on this just to know what others feel. The poll is in English ;-D

    ReplyDelete