Not all polls are created equal
| Economy dominates final days of race |
By Mark Z. Barabak
Los Angeles Times
Every day dozens of polls on the presidential race are published, taking the measure of voter sentiment nationally and in key states. Depending on the numbers, Barack Obama is headed for an electoral vote landslide Tuesday, or John McCain has a shot at yet another come-from-behind victory.
Obviously, both can't happen, which suggests that at least some polls are askew. Take, for instance, Nevada, a state that Obama hopes to win as part of a Democratic incursion into the conservative-leaning Rocky Mountain West. One poll this week put Obama's lead at 12 percentage points. Another gave the Illinois senator a 10-point advantage and still another a 7-point lead. Two others said Obama's lead was 4 percentage points, meaning McCain, a senator from Arizona, could be ahead slightly given the margin of sampling error.
Why such a big difference in polls conducted in the same state over roughly the same period?
There are several reasons, some having to do with the inherent nature of polling, others with factors specific to this highly unusual presidential campaign, which has given fits to even the most experienced pollsters.
Opinion surveys are based on statistical probabilities. The idea is that by interviewing a representative sample of voters, a pollster will achieve the same result as if each American voter had been interviewed.
While some are skeptical of that fundamental premise, the pioneering George Gallup had a ready retort: "An accurate blood test requires only a few drops of blood." In other words, a pollster can attain a reasonably accurate gauge of how 100 million or more Americans will behave by conducting a scientific sampling of 1,200 or so voters.
But there are any number of reasons that polls come up with varying results. Sometimes questions are worded differently or posed in a different order. There are also different ways of choosing whom to sample. Some polls will talk to individuals at random. Others work off lists of registered voters. The age, gender or ethnicity of the person asking the questions can affect the response.
For that reason, some pollsters employ interactive technology, using a recorded voice or the Web. Others, however, frown at that because there is no way to know whether the respondent is a voter or their 6-year-old child. Any and all of those factors can cause results to differ.
Question: So how do pollsters know they are interviewing a representative sample of voters?
Answer: That's where art and science come together. A pollster will attempt to determine who among those interviewed are the most likely to vote. This year it's especially tough to define a "likely voter" given Obama's particular appeal to black voters and young people, two groups that typically fail to vote in numbers commensurate with their share of the population. Different pollsters have different ways of determining who they consider a "likely voter." For instance, one recent national survey that showed the race neck-and-neck included a large number of born-again and evangelical Christians, which improved McCain's performance.
Q: What else is important in assessing polls?
A: Timing is crucial. Polls taken before or after a significant event can vary considerably. This year, the selection of the candidates' running mates, the two major party conventions, the presidential debates and the crisis on Wall Street all affected public opinion, especially in the short term.
Q: What do pollsters mean when they talk about "a margin of error?"
A: Because they are not talking to every voter, pollsters recognize there is a certain squishiness in their numbers. This "sampling error" is measurable, based on a standard statistical calculation. (Rule of thumb: The bigger the sample size, the smaller the margin of error.) Jill Darling, associate director of the Los Angeles Times Poll, explains: "If Smith has 52 percent of the vote and we have a margin of error of plus or minus 3 percentage points, that means that if everyone voted right now, Smith would get between 55 percent and 49 percent of the vote. And if my survey finds that Jones has 48 percent of the vote, then his actual vote would be somewhere between 45 percent and 51 percent." So to say that Smith is ahead, his lead would have to be twice the margin of error, or more than 6 percentage points. That is why a poll showing Smith at 52 percent and Jones at 48 percent means the race is about even.
Q: Any other suggestions for sorting through the polls?
A: Neil Newhouse is one of the country's leading Republican pollsters. He also conducts, in tandem with Democrat Peter S. Hart, the NBC/Wall Street Journal poll. When it comes to assessing polls, Newhouse said, "Throw out the high score, throw out the low score and average the rest." (Several Web sites offer a running sample of surveys, including www.RealClearPolitics.com, Pollster.com and www.fivethirtyeight.com.)
Perhaps the key thing to remember is that even the best poll can do no more than capture sentiments at a given moment.