If they keep quiet, the stones will cry out." Luke 19:40

Stones Cry Out: What Went Wrong with the Exit Polling?

Two days after the election, Dick Morris wrote regarding the exit polling,

This was no mere mistake. Exit polls cannot be as wrong across the board as they were on election night. I suspect foul play.

At the time, I didn’t really know what he was talking about. My “eye” told me that the data posted on CNN’s web-site were highly accurate when compared to the election outcome. I assumed that Dick’s problem was with the early release of the exit poll data and that perhaps these early data were not appropriately weighted.

Then I read Richard Morin’s piece in the Washington Post yesterday blasting bloggers and their complicity in the dissemination of the early exit polling data. For the most part I agreed with Mr. Morin and posted a link to his article. However, his opening perplexed me.

It will be a few more weeks before we know exactly what went wrong with the 2004 exit polls.

There was that charge again!

From the context, it seemed Mr. Morin was saying something was wrong with the final exit polls, not just the interpretation of the early returns. I decided to test the exit polls against the election outcome. I gathered all the CNN exit poll data for the nation and the battle ground states and loaded it into a spreadsheet. Then I keyed in all the election results for these geographies and performed a simple statistical test for determining whether a proportion derived from a sample (exit poll) is significantly different than an established standard (election result).

According to this test, the exit polling appeared to be remarkably accurate!(Exhibit 1)

As we know, the election result was approximately 51.0% Bush/48.1% Kerry. The nationwide exit poll reported by CNN appeared to predict that Bush would win the popular vote 51.2%/47.9%. Given that the sample was based on 13,660 voters, I was not too surprised by the “accuracy” of the exit poll. A sample this large should yield predictive results accurate to within 1% margin of error with roughly 97% confidence.(1)


Blogger Rick Brady said...

"A sample this large should yield predictive results accurate to within 1% margin of error with roughly 97% confidence.(1)"

I appreciate the link, but I must warn you that the "(1)" is a footnote that indicates this statement is not entirely accurate. This statement assumes a simple random sample, but in fact exit polls are based on cluster samples. This means a higher margin of error and lower confidence level. Therefore my findings presented in this table are a bit exaggerated. However, I think the essence of my spreadsheet is correct. The exit polls were heavily biased towards Kerry.

The question now is why?

11/28/2004 06:56:00 PM  
Blogger lawnorder said...

After reading Rick's blog I realize that unlikeme, he is NOT a rabid Kerry supporter. Yet he is still troubled by the exit poll discrepancy. Interesting!

Rick calls the discrepancy a skewing towards Kerry on exit polls. I call it a skewing towards Bush on the e-vote machines. Potato, Potatoe...

What matters is that both sides are puzzled enough by this discrepancy. We should get a trustworthy bipartisan team to review it. But after the 9/11 and WMD commission whitewash my hopes of seeing any serious bipartisan effort are slim...

11/29/2004 07:51:00 AM  

Post a Comment

Links to this post:

Create a Link

<< Home