Perhaps the biggest challenge for a researcher is to show causality – that X is causing Y. Most of the time, at best we can only claim that the two are correlated. Even if we do some fancy econometric techniques, we need to be careful in making claims about the coefficients. Usually, we phrase our conclusion as “higher/lower X is associated with higher/lower Y.” We tend to avoid saying things like “higher/lower X is causing higher/lower Y” due to reverse causality and omitted variable biases.
(Even if we just claim an association, we also need to be careful. Otherwise, we’d say things like "people born under the astrological sign of Leo are 15% more likely to be admitted to hospital with gastric bleeding than those born under the other 11 signs," as this article quoted.)
If" we want to claim causality, we need to think about counterfactual: “what would have happened to Y if X had not existed or happened.” Think about X is a policy or any kind of intervention, and Y is the outcome. Let’s say that X is classical music, and Y is baby’s IQ. Some people said that classical music increases (that is, causing in a positive way) baby’s intellectuality. Truly, I don’t know how true the research was; perhaps it was, perhaps it wasn't. The point is, all the researchers need is to provide a counterfactual: that babies who did not listen to classical music have lower IQ scores that those who have been exposed to classical music.
But simply comparing two babies (or two populations of babies) will not solve the problem. We may think that parents’ taste of music is correlated with wealth. Hence babies who were exposed to classical music are more likely to come from wealthier parents who can feed them with better nutrition or supply them with more IQ-stimulating games. This is what we call ‘omitted variable bias.’
A perfect research would be making a clone of a newborn baby, treat them equally the same except for the exposure to classical music. Then measure their IQ after several years. Note that a research involving human cloning will not pass the ethical committee, at least until now. The next best thing we can do is to perform the so-called randomized experiment (like these guys, as well as this friend of us have been doing).
But Dhani (the musician, not our friend Dhani the real researcher) had a different idea. In his latest appearance in an infotainment (indeed, watching infotainment is better than listening to football commentator during the match break. By the way, Dhani appeared with his three boys but without Maia), he said that his boys must like rock music, because it is a ‘man’s music.’ His conclusion was based on a ‘study’ over some ‘sample’ – his so-called ‘sissy, queer’ (bences-bences) friends. His ‘sample,’ he claimed, do not like rock music. So look at the result. (The presenter asked Dhani’s kids whether they listen to Dave Koz, which answer was ‘no.’ Dhani then added, “Of course they don’t listen to him – they like Iron Maiden instead…”).
I enjoy Dhani’s music. I respect him as a musician. I never like his male-chauvinist antics and remarks. And definitely he is not a researcher.