It Depends How You Ask the Question

A couple of days ago at NRO , the estimable Michael Barone ruminated on the subject of same-sex marriage.  He casually professed himself in favor of this revolution in the institution of marriage—giving an extremely bad reason that suggests he has not thought a great deal about it—but for the most part he restricted himself to analyzing where our politics is headed on this issue.  On political analysis, there are few who can equal Barone, yet he ended on a curiously indecisive and unsatisfying note.  He seemed to be saying that same-sex marriage ought not to be any great concern in the upcoming presidential election, because we have this “experiment” going on in half a dozen states right now, and we should be content to see how it works out, for now, without national politicians getting into it.  Perhaps the explanation for this thumb-twiddling recommendation is that Barone, whose political sympathies are visibly right of center, believes same-sex marriage is a losing issue for Republicans in national politics, because the revolution about which he is so insouciantly accepting is “inevitable.”  At least that is a fair inference from his column, though it is not explicitly his argument.

Then yesterday, at Public Discourse , the indefatigable and razor-sharp Maggie Gallagher had her innings in PD ‘s two-week series on the fundamental issues that confront us in the 2012 presidential election.  Same-sex marriage is something Maggie has thought about more than nearly anyone, and she made a compelling case that the issue belongs in the forefront of any conservative candidate’s concerns, and that the defense of marriage is a political winner.

How do two such smart people—I have been in the company of each and I can tell you it’s taxing to keep up—come to such different views of the same issue?  Aside from their stated preferences on the issue—Barone limply in favor of same-sex marriage, Gallagher ferociously opposed—they relied on different sets of factual evidence.

Barone led with the Gallup poll, which reported in May that for the first time since it began surveying on the subject of same-sex marriage fifteen years ago, a majority of respondents declared they approved of it.  In 1996 the responses were 27% in favor, 68% opposed; in May of this year they were 53% in favor and 47% opposed.  As Barone remarked, on most issues, change in public opinion is far more more gradual if it shifts at all, so this is big—and he finds it mostly explained by the high numbers of young people who approve of same-sex marriage.

Gallagher had at her disposal a quite different set of survey results , obtained in the same month by the firm Public Opinion Strategies (commissioned by the Alliance Defense Fund).  That poll found only 35% who seemed to accept same-sex marriage, with 62% opposed (I’ll explain that “seemed to” in a moment.)

That’s quite a significant difference between two national, large-sample public opinion surveys conducted in the same month (POS had a larger sample, and a smaller margin of error, than Gallup, but both employed respectable methods).  But look at how each asked about the issue.  Here’s Gallup: “Do you think marriages between same-sex couples should or should not be recognized by the law as valid, with the same rights as traditional marriages?”  Until 2006, “same-sex couples” was “homosexual couples” but otherwise this is the question Gallup has been asking since 1996.

In POS’s survey, respondents were asked to agree or disagree with this statement: “I believe MARRIAGE should be defined ONLY as a union between one man and one woman” (caps in original, for emphasis when interviewers read the statement aloud).  Respondents who said they agreed or disagreed were then asked whether they “definitely” or only “somewhat” held the view they had expressed.  We may take the 35% who disagreed as seeming to support same-sex marriage—but there may have been a few who disagreed because they approve of polygamy.  All we know is that they disagreed with the statement.  And we know they were vastly outnumbered by those who agreed with it—again, 62%.  And when pressed further, 53% said they “definitely” agreed, not just “somewhat.”

There is something plain-spoken and direct about the POS question, and something very useful about the follow-up on the intensity of opinion.  Gallup’s question, on the other hand, is actually a bit ambiguous.  Ordinarily in this business, one likes to keep asking the same form of the question year after year, yielding data for a valid comparison.  But in in this case, it may be that Gallup’s question has been rendered obsolete by events.  (The survey already ditched “homosexual marriage” for “same-sex marriage” for obviously politically-correct reasons, but that’s not my point.)  In 1996, “Do you think marriages between [homosexual or] same-sex couples should or should not be recognized by the law as valid . . . ?” was entirely a hypothetical question.  There weren’t any such marriages, and so respondents could only understand this question to mean, Do you think the law should permit two men or two women to marry each other?

Now, of course, there are same-sex couples getting married in the eyes of six states (five at the time of the survey, before New York changed its law), and the question may take on a different aspect.  There are marriages between same-sex couples.  Would you have them “recognized by the law as valid,” or not?  This is a confusing question for nice, ill-informed people.*  A substantial number of them may think that they are being asked whether those gay couples married in another state ought to have their marriage “recognized as valid.”  They may not at all want the law on this question changed in their own state—and they may not even be thinking clearly about “recognized as valid” as something they really want to happen in their own backyard with respect to marriages contracted elsewhere. 

It’s just not a clear question, in the current legal landscape, and has too many gopher holes to trip in.  At its best, it might say something or other, not very clearly, about the issues surrounding DOMA and interstate recognition of same-sex marriages.  I’m not sure it validly says anything, if it once did, about the basic issue of permitting men to marry men and women to marry women.  And so the purported “shift” in opinion since 1996 may be, to a substantial but unknown degree, owing to this erosion into ambiguity of Gallup’s question.  On the basic issue, the POS question is vastly superior for simplicity, clarity, directness, and a measurement of intensity.  And the results at the real polls—the ones where people vote whether to defend marriage or not—bear out POS’s greater probability of accuracy.  In 31 states, the people have gone to the polls and said yes to marriage and no to the destructive revolution represented by same-sex unions.

I’m with Maggie Gallagher, and I think we’re both with a clear majority of Americans.  Marriage is between one man and one woman, and a presidential candidate who keeps saying that can only help himself.

*And make no mistake: Gallup talks to lots of ill-informed people, as all pollsters invariably do.  In the same month as the poll we’re discussing, Gallup found that the average answer to the question “how many gays and lesbians are there in the U.S.?” was that they were one quarter of the population—roughly a tenfold error.  Numbers were highest among those least informed respondents—the young—who were also the most inclined to approve of . . . something or other on the subject of same-sex marriage.