Saturday, February 7, 1998

We don't live by polls alone

By DALE HANSON BOURKE / Religion News Service

(Dale Hanson Bourke is author of "Turn Toward the Wind" and publisher of Religion News Service.)

UNDATED -- First, let me say I have nothing against pollsters. They are, for the most part, mild-mannered statisticians who love to count anything coming across their path.

What does bother me is the misuse of polls and the way even the most questionable research is elevated to headlines. Even more frustrating is the way the media seems to run after public opinion polls, letting them shape the content and amount of coverage.

The cascade of polls -- many of them snap, overnight testings of opinion -- conducted in the midst of the controversy surrounding the Clinton White House are a prime example of the pitfalls of polling.

Having once worked as a lowly market researcher, I may be jealous of the new status granted to those who measure the prevailing winds of opinion. But I also know enough about research to be suspicious of its usage and cautious about extrapolating generalities from specific responses.

First, we must understand a survey is only as good as the instrument used. The instrument, in this case, is the questionnaire. Under most research conditions, questions are pretested to make sure certain wording, placement and other unintentional cues don't skew the results. It is critical to accurate research that questions be posed in as neutral a fashion as possible.

Most of the polls reported in the last week were done so quickly it is hard to imagine the questions themselves were properly tested. What makes the reports even more confusing is the fact that the actual questions are summarized rather than restated.

What the viewer of a television report sees is a snapshot of a question with percentages of answers. Was the question the only one asked? Or did it come after several others that tended to color the thinking of the person? For example, if the pollster asks several questions about morality and then asks about job performance, the person responding will tend to separate the two issues, feeling he or she had already dealt with the moral implications.

Perhaps the most important information that needs to be understood appears in the fine print at the bottom of the poll report -- the sample size and margin of error.

In the past, this data was always verbalized in broadcast reports because of its critical nature. Lately it seems only to appear in small print on the screen.

Sample size and margin of error are indications of how much one can trust the poll information. The sample size is easy to understand. Many of the polls purporting to speak on behalf of the entire American voting public are taken from interviews with a few hundred people. While this seems like a very small number, it can be considered fairly accurate if the people are truly random selections.

With the short amount of time between the commissioning and completion of many of these recent polls, I wonder if a representative sample can truly be found.

It's also important to take a look at the margin of error. Many of the recent polls, for example, have shown a swing of as many as 5 percentage points.

In my research days, a 3 point swing was the highest allowed. As one of my professors urged, "Add that number to the lower percentage, subtract it from the higher, and then see if you have anything to talk about." The fact is a 5 point swing in either direction renders moot the conclusions drawn from some recent polls.

The greatest danger is that journalists and politicians will chase polls instead of using them to shed light on issues. When a poll shows people are tired of hearing about a subject, should journalists stop reporting on it? When a survey shows the public will vote a certain way, should a smart politician align himself with popular opinion?

Both Journalists and politicians have leadership roles in popular culture. A journalist is supposed to shed light on news and issues without thinking about whether it sells newspapers or raises ratings. Politicians are supposed to campaign on platforms embraced for conviction, not popularity.

Public opinion polling is a tool. But leadership in our society is sorely needed by those who are in front of the pack, not behind it.

When public opinion polls show people are tired of both journalists and politicians, perhaps both groups need to consider that people are simply weary of leadership that lacks conviction.