Michael J. New highlights a methodological limitation in a recent abortion study promoted by Amanda Marcotte and others:
A medical student contacted the authors of the study to ask how they obtained pregnancy and abortion data from study participants. As it turns out, the researchers used telephone surveys. Overall, this is extremely problematic. Many women who submit to an abortion will not voluntarily reveal that information. As such, this study likely undercounts abortions and overestimates the effectiveness of contraceptives. This is obviously a significant methodological limitation to the study — one no mainstream-media outlet has yet to cover.
This is a reminder of how careful one must be in citing sociological data. If our beliefs are correct, we can expect reality to conform to them, but working the other way is harder if not impossible. We won’t strike upon first principles or basic commitments while sifting through fine-grained data. The real argument will always be deeper.
October 31st, 2012 | 9:53 am
Don’t assume that telephone surveys that ask sensitive questions are plagued by social desirability bias. Any good telephone survey would use telephone audio computer-assisted self-interviewing (T-ACASI) to reduce or eliminate bias. In T-ACASI, the respondent pushes buttons on their phone to respond to sensitive questions. I skimmed the article, and it was unclear as to whether they even did a telephone survey. The study does have its methodological limitations (creation of control group, etc), which are mentioned in the discussion section of the article and are highlighted by other bloggers.