Why I don't link to the TT/UT internet poll

Yesterday I mentioned that as a matter of general policy, I don't post polls of house districts for statewide races. Today's post is why I don't report on the UT/Texas Tribune polls.

The UT/Texas Tribune poll is an internet poll, based on opt-in. It is difficult to understate the fundamental theoretical difference between an opt-in internet poll and a telephone poll. The sample is not random.

I wish I could simply cut and paste from UT/TT poll methodology page, but frankly I found their discussion hard to understand, and I know how the poll works! So, at the risk of minor errors, let me try to simplify: the UT/TT poll is done by PoliMetrix, which obtains consent of participants (they find them through Google Ads) and keeps them in a database (currently about 55k Texans). When they need to do a specific poll, they take a random sample of the registered voters provided by the Secretary of State. Then, they attempt to match the people that were randomly selected from the voter pool with one of the people in their database who has already consented. They use an algorithm to find someone who matches the randomly selected voter as closely as possible on the basis of race, age, gender, region, and partisanship. Then they send them a web survey (if they get no response, they match to a similar person), compile their results, and voila! a poll.

The most recent study led the authors to conclude that the emprical data does not say that "opt-in sample surveys are as accurate as or more accurate than probability sample surveys." There was some back and forth (not an exhaustive set of links, obviously) which argued many things, but including if the additional margin of error was really significant.

Polling is, by nature, a cost/benefit analysis. Polling attempts to save money by taking a sample which saves money by alleviating the need to ask every single voter. Internet polling is cheaper, and has managed a decent track record, if not quite what telephone polling has. [Sidenote: I supported automated (IVR) polling before it was commonly accepted, because it was obviously predictive. So it's not that I'm reflexively anti-new forms of polling.]

However, with that said, I doubt that any professional pollster of either party has ever used an online poll for campaign purposes. As of now, telephone polling is more accurate. While it's a small sample size, PoliMetrix's polls in the 2009 races had Corzine 43, Christie 41, McDonnell 53, Deeds 40. McDonnell won 59 to 41, and Christie didn't actually lose, he won 49 to 44. While only eyeballing, I'm pretty sure that's worse than your average poll. It might also suggest that PoliMetrix underweights Republicans, though significantly more data would be required.

In sum, internet polling has improved. The UT/TT poll by PoliMetrix is on the borderline for me. In the future, it might be undebateably acceptable for two reasons: 1) it proves itself to be equal in accuracy to telphone polling, and 2) due to cell phones, telephone polling declines. And, if we were in a period with no other data, I would probably mention the UT/TT poll. But right now we some regular and irregular polling be done by established polling organizations, so in my judgment the poll doesn't add to what we already know.

Posted by Evan @ 02/14/10 01:05 PM


Previous Entry | Home | Next Entry


No comments yet

Add Comments

No flames or impolite behavior. HTML will be stripped. URLs will be transformed into hyperlinks.