by Tony Petrangelo

Jul 23, 2012, 7:00 AM

Rule Number One of Hitchhiking and Poll Watching, Don’t Panic

I woke up Saturday morning only to immediately be confronted by the gnashing of garments and rending of teeth over the seemingly impossible results of the new SurveyUSA (KSTP) poll on the hatin’ on marriage amendment.

There must be something wrong with the poll right? I mean, the last two polls, one by PPP and one by PSRA (StarTribune) both showed the amendment losing, how could there be such a dramatic shift in such a short time? Both of those questions are versions of questions people were asking.

The simple reality is that this poll isn’t a whole lot different than the previous poll that SurveyUSA conducted in the state back in February. Take a look:

SurveyUSA (7/20, 2/10 in parenthesis):

An amendment to the Minnesota Constitution on the ballot defines marriage as between one man and one woman. Will you vote FOR the amendment? Against the amendment? Or not vote on the measure?

For 52 (47)
Against 37 (39)
Not Vote 5 (10)
Undecided 6 (4)
(MoE: ±4.3%)

The big movement is the five point shift in favor of the amendment and the five point shift in the number of respondents who said they would “Not vote” on the amendment. If you look “under the hood” of the poll, you can see what is likely the primary driver of the difference, and that’s the partisan ID.

In the poll from February, only 23% of respondents self-identified as Republicans, while in this latest poll that number is 32%. You also see a big change in support by age, with the 18-34 cohort opposing the amendment 44-38 in the previous poll and supporting it 50-44 in the recent one.

Regardless of the composition of the electorate in November, I don’t think that anyone thinks the amendment will win among the 18-34 age group.

There is a difference between these two polls though, the older one is a poll of registered voters while the newer one applies a likely voter screen. But is it too early to use a likely voter screen and is this what is causing the apparent shift in favor of the amendment?

Nate Silver recently wrote an article addressing this very subject. Here is one of the very learned things he had to say on the matter:

Many polling firms believe that likely voter models are unreliable when applied this early in the campaign, and most avoid doing so. This evidence suggests that this is a solid intuition: the things that get voters excited in July may not be the same ones that wind up doing so in November.

The “This evidence” he’s referring to is the study he did, as outlined in the article. His research shows that likely voter screens become more reliable than registered voter polls after Labor Day, but that at this point in the cycle they are actually less reliable.

But, his research also shows that on average, the difference between a registered voter model and a likely voter one only helps Republicans by about 1.5 points. The shifts in this poll are more substantial than that, but within the bounds of the polls margin of error when you consider the change to a likely voter screen.

In much the same way that this poll seems to skew towards the side of amendment supporters, the same thing could be said about the PSRA poll from May but for the other side. And if you’re in a mood to find exception with this SurveyUSA poll, exceptions should also be made with the PSRA poll.

The PSRA poll asked the following question:

Please tell me if you would favor or oppose amending the Minnesota constitution to ban same-sex marriage.

But there is nowhere that the amendment question itself ever mentions same-sex marriage, so this is a faulty question.

This SurveyUSA poll shows 52% in support, the highest number we’ve seen this year, just as the PSRA poll showed the highest number of any poll in opposition to the amendment, 55%.

Looking at those opposing the amendment in the SurveyUSA poll we see the lowest number of any poll, while in the PSRA poll we saw the lowest number for support of the amendment.

If you were to simply average these two polls together, you get a 46%-46% split. Which is pretty much where the overall polling average is right now.

It’s possible that this poll is an outlier, it’s also possible that it’s not, but either way, it’s just one poll. That’s why a simple average of available polls will almost always be a better indicator of the state of play in a given race than any one poll on it’s own.

Nate Silver, the very learned gentleman quoted and linked above, wrote a whole series of posts on going over the accuracy of polling averages in predicting election outcomes.

Want an example? The final polling average I published before the Wisconsin recall election showed Walker winning 51%-46%. The final results: 53%-46%.

I didn’t do anything special there, just put all the poll results in a spreadsheet and averaged the numbers and bam, a pretty darn accurate prediction.

For the amendment I have two different versions of the polling average, the first one breaks the four different groups of respondents (Yes, No, Abstain, Undecided) out and the second combines the Against group with the Abstain group, as a Non-Vote will count the same as a No vote.

What you can see is that right now things look really close. When you add the non-votes to the no votes than the amendment actually fails in three out of the five polls in the database.