The public opinion poll has become an ubiquitous feature of modern life. Seventy years ago, there were no professional pollsters. Fifty years ago only a handful—Gallup, Roper—served as takers of the public pulse. Today, thanks to computer and telephone technology, thousands of public opinion seers and sages are for hire. The explosion of practitioners is only the most visible change. More important is the poll’s new authority and legitimacy. With its esoteric technical terms, data banks staffed by skilled technicians, complex statistical procedures, and close associations with prestigious academic and business enterprises, it is scientifically authoritative.
Even if this were insufficient, polling is fully cloaked in democratic legitimacy—democracy means heeding the people, and what better way to give an ear than through the poll? All political persuasions embrace it. Predictably, such an available and powerful tool is relentlessly applied. No self-respecting official, candidate for office, government agency, opposition group, or mass media purveyor can afford to be without one. To say that the modern poll has become the scientifically sanctioned, consensually celebrated form of self-revelation and self-understanding is no exaggeration. When the vox populi speaks through the pollster, as it always seems to be doing, everyone listens reverently.
A few still hold out against this mechanical peephole into the public mind. These nonbelievers still prefer their insights the old-fashioned way—semi-rambling conversations with bartenders, cabbies, and barbers (not hair stylists). To them, the whole bean-counting endeavor is beyond salvation; it is pointless even to read the polls.
But such Luddites are rare. Far more common are the uneasy skeptics, those who occasionally sneak a peck, especially when findings confirm intuition, but who cannot truly believe. Deep down, the endless parade of cold numbers makes them nervous. Their resistance to conversion is vague, ambivalent, not rooted in a hard-data counter-vision or principle. They cannot decide whether polls offer passing curiosities, harmless entertainment akin to “Ripley’s Believe It or Not,” or whether they capture some sliver of reality.
We address a simple question: leaving aside dishonesty and technical incompetence, can the skeptic generally accept polls? Our answer is also simple: polls occasionally reveal a modest portion of reality, at best. It is not that people consciously lie, though some do. Nor is it that the head counters are schemers exploiting science, though some are. Rather, the polling process by nature cannot present a full, well-rounded picture; a murky virtual reality is more like it. Polling may be compared to a particular hand tool—it does a few jobs well, other tasks adequately, but you cannot build a house with a screwdriver.
What polls can build is public opinion. At a minimum, the term “public opinion” predates by two centuries the modern poll. “Opinion of the people” appeared in England as early as 1741; esprit public and conscience publique emerged in late 18th-century French writings. The absence of survey technology did not, of course, preclude discovering public sentiment. Opinion was displayed in newspapers, polemical writings, speeches, crowd reactions, public demonstrations, and numerous other visible manifestations. Visiting the local coffeehouse, salon, or Tischgesellschaften might suffice. Periodically, public opinion was self-evident, as, for example, when an angry mob dispatched the tax collector. When the public spoke, it did so without pollster midwives.
Today, matters are vastly different. Benjamin Ginsberg’s The Captive Public depicts how these spontaneous revelations of popular sentiment are now crowded out by the more authoritative scientific poll. To fathom the public’s mind on, say, taxes, one collects available poll data or commissions one’s own statistics. These data, not irate letters to the editor, calls to radio talk shows, organizational activity, magazine readership, or widespread tax avoidance, comprise public opinion. All else is inferior merchandise, the stuff cherished by the preindustrial Luddites. Scientific numbers defeat informed acrimonious public debate and activity.
The consequences of this shift are profound. Simply put, no public opinion exists without it being collected and displayed. The poll, not what people say among themselves, is public opinion; it is manufactured, not spontaneously revealed. The priestly Oracle attendants look suspiciously like ventriloquists. Citizens could be thinking about mass insurrection, but this sentiment cannot “exist” unless captured by the poll. Why use force to silence citizens when it is easier to manipulate poll questions? For example, almost since the advent of polling, people have been endlessly queried about taxes, usually about whether they are too high. Questions addressing the system’s very structure—the federal income tax and mandatory withholding, for example—remain unasked. Obviously, poll maestros do not judge such questions as worthy of investigation.
The way polling shapes public opinion goes beyond the nature of the questions asked. Control is pervasive, including interpreting the final results. Even when information is collected, distribution and publicity are not required. A survey revealing that most whites believe blacks to be genetically inferior could be hidden away for “future use.” Conversely, matters of modest interest can be raised up to the level of “public concern” simply by including questions on a widely distributed survey. For example, between 1973 and 1989, the National Opinion Research Center asked 11 questions on homosexuality. Other survey organizations likewise discovered homosexuality. These fresh data thus interjected a new item into the soup of public opinion. Suddenly, the public had views on the subject that could be criticized, celebrated, or even followed.
“Public opinion” now more closely reflects commercial considerations or academic fashions, not necessarily what citizens rate critical. Remember, public views are now manufactured, not spontaneous, and this production is not performed pro bono by disinterested altruistic philosopher kings. Nor do the local folk finance polls from bake sale proceeds. Polling is an expensive product that, ultimately, must be sold—to the media as news or to those who sponsor large-scale research, especially government funding agencies. Given what we know about elite agendas, a certain tone is to be expected. Polls tell us a great deal about resistance to busing; less about what happens in racially integrated schools. Perhaps poll sponsors do not want to hear about this. Questions regarding more government services and regulation almost always make “benefits” highly tempting. Remember, no questions, no public opinion, and he who controls the poll, controls public opinion.
Public opinion is always measured using samples. Experts agree that a sample of about 1,500 randomly selected, physically dispersed people will, in a statistical sense, “represent” a larger population. It is an atomistic view of society: public opinion is simply a collection of individual opinions. The possibility that public opinion must emerge from collective deliberation, that it also may be a property of groups and interests, is definitionally impossible. The modern poll rests on a vision of mass society. But, leaving aside this uncase, how is “public” defined in public opinion? Not statistically, but substantively.
This question, an issue of momentous political consequences to our communal self-definition, is seldom addressed. Resolution is by the technical, mechanical dictates of sampling: any conscript claiming to be over 18 becomes officially enrolled in our public. “The public” is, de facto, identical to “the sample.”
This modern, convenient, and indiscriminate approach departs from tradition. Previously, inclusion in “the public” required some connection to the agenda. A connection might be rooted in self-interest, involvement in events, legal status, position in society, expertise, or any other quality authorizing participation. Demarcation lines were not necessarily precise or strictly followed, but somewhere a division between legitimate participants and outsiders existed. Gate-crashers would be ignored, no matter how noisy. Today, of course, the club of public opinion, like pre-approved MasterCard applications, accepts anyone who breathes. If, per chance, the pollster approached illegal immigrants awaiting deportation, he could freely enlist them as bona fide members of the American public. Unless specified in the questionnaire itself, their responses regarding extending citizenship and benefits to non-citizens are valid. This point was driven home to me when my 13-year-old son was telephoned by Newsweek for its final 1994 pre-election poll. Identifying himself as a 24-year-old, he waxed eloquent on national affairs.
This porous, unreflective definition of “the public” raises several troubling issues. For example, do we welcome those incapable of offering reasoned opinions? Taken together, these would-be pontificators are a significant slice of humanity: alcoholics, drug addicts, the senile, the retarded, the severely mentally ill, recluses, the gravely disabled, and others constituting the public in body only. Should we take seriously advice from incarcerated felons, guest workers, foreign students, and visiting tourists? Must pollsters reach out to those wholly within ethnic enclaves lacking fluency in English? There are no rules in the polling industry. Critical interpretations of political community are almost casually determined by cost, convenience, and the easy availability of respondents. You take who is available.
What is troubling here is not the remote possibility that pollsters may stumble into Bedlam seeking guidance on nuclear deterrence. What is far more common in the typical semiconscious sweep of census tracts and phone numbers are recruits whose contributions are, essentially, irrelevant. Some initiative was once required to join the public—one might, for example, have to defend one’s view in public, demonstrate knowledge, or risk one’s reputation. Obvious fools could be ignored. A degree of responsibility was thus imposed—the admission ticket was standing and contribution. Today, of course, enrollment is passive and anonymous—the pollster comes and gets you. A substantive connection, stake, or even interest in the issue is assumed, rarely demonstrated. The reality is that most people, most of the time, have no business offering snap judgments on complex, distant matters that are none of their business. What authorizes the average citizen to speculate on intervention in Bosnia or wetlands policy? Such counsel, given without fear of rebuke, easily becomes irresponsible. The “right” to participate in public debate is granted solely by geographical location—those who happen to fall into the sampling frame. If the typical sample were physically assembled for a good public debate, many, no doubt, would be hushed up or asked to leave. They had no legitimate business there. But, when converted into statistics, their voices become authoritative.
Indeed, genuine contributors may disproportionately opt out of the polling process. Being thoughtful souls, they find the typical simpleminded poll question a waste of time. If they do respond, all the careful qualifications, asides, and other insights wind up on the cutting room floor as detailed answers are compacted into “yes” and “no” boxes. As in the selection of jurors, the process is biased towards the unsophisticated at leisure. A republic of not-too-busy TV watchers.
Clearly, even the most ardent defenders of polling acknowledge a degree of untruth in their product. Surely people lie, especially on sensitive matters, and surely surveys occasionally employ ill-defined terms that elicit nonsense. Questions omit relevant alternatives, dramatic events distort reactions, and bumbling interviewers garble responses. Furthermore, results are sometimes distorted by unrepresentative samples, clerical errors, or even outright falsification. This is all openly confessed, though dismissed as nonfatal. Let us not dwell on these familiar weaknesses; our concerns run deeper.
In the real world, misjudgments and misperceptions are costly. While market research polls better get it right or heads will roll, this is not true when assessing what citizens want from government. Even more important, there are few incentives to get it right. If concocted results were placed next to the genuine article, how could we distinguish the gold from the base metal? There is no standard “truth bar” in some Bureau of Opinion, and here lies the problem. When a New York Times survey reports that 75 percent of the public want less defense spending, and the Platonically correct figure is 49 percent, does anyone lose his job? Arc pollsters panicked that the public will lose confidence in the pay-per-view Oracles? Of course not—nobody (fortunately?) has the Platonic figure. We have assertions of accuracy buttressed by scientific paraphernalia, not public tests.
In general, tests of poll accuracy on matters of public preference are exceedingly rare. Perhaps the only example is the pre-election poll. This test, however, is somewhat limited. Voting, unlike policy, has a concrete meaning to respondents, while the threat of accountability permits pollsters to take precautions such as larger samples and multiple questions. Even when predictions miss, escaping responsibility is not difficult. There arc always last-minute disturbances, “understandable” limitations of technique, and other excuses. In especially difficult situations, pollsters announce “too close to call.”
The very nature of polling offers few incentives for getting closer. All survey organizations, whether small campaign shops or university-connected colossi, are constrained by costs. Improving accuracy is very expensive—intensive interviewer training, numerous pretests, multiple questions, and other quality control steps inflate costs. And one can still never be sure. Why should the market—the mass media, politicians, or academics—pay handsomely for this enhanced quality? A better poll question is not the same thing as buying a Lexus versus a Chevrolet. In short, there is no demand for the better product.
Perhaps the most serious and enduring impediment to accuracy is that nobody can tell the difference, at least for questions about political preference. We are shooting at an invisible target. There are no full-page ads in the Wall Street Journal proclaiming that, for the fifth year in a row, Gallup has been certified to have the most accurate polls. Accuracy will come only when we have a clear standard of true sentiment, and this is likely to be never.
The modern poll assumes that the people have something intelligent to say. Otherwise, why ask? Among both commercial pollsters and academics, this must be judged The Supreme Article of Faith. To challenge it attacks the very foundations of the enterprise. Occasional stupidities will be dutifully acknowledged, commented upon, picked apart in print, and everybody can then return to work as if nothing happened. The problem is deeper than acknowledged, however.
The typical survey’s very design conspires to maintain the faith. Basically, it is a highly structured arrangement that could transform even noises from the brain dead into valid data. Merely by answering “yes” or “no,” citizens “resolve” the most complex, troublesome problems. It is assumed that by simply reacting, respondents know of what they speak. For example, one common type of question is, “Would you increase/decrease/ or keep the same spending on policy X (defense, social security, environmental protection, etc.).” Despite studies demonstrating vast ignorance of government spending, the respondent’s knowledge is never assessed. Hence, Mr. X, unknown to the pollster, believes we currently spend $800 billion on defense and advocates a “decrease” to bring it down to a mere $500 billion. Meanwhile, Mrs. Y is certain that the present figure is $200 billion and wants an “increase” to $300 billion. Finally, Mr. Z is well-informed of the facts, including the difficulty of precisely labeling “military spending,” and favors a marvelous plan of both cuts and increases. He reluctantly answers, “Don’t Know.” The headline declares “Public Divided on Defense Cuts.”
Simpleminded questionnaires mask the public’s inability to navigate difficult choices, especially when large numbers and rare events are involved. Only occasionally does collective silliness expose itself. For example, respondents in one survey expressed a willingness to impose a $10 tax on Westerners to save 50,000 birds in an oil spill. This is $2,000 per bird! This is peanuts compared to the $32 billion a year wanted to preserve the whopping crane and $244 million to prevent a single offshore oil spill. But, public generosity does have its limits. Polls on willingness to pay for government-sponsored health care show most Americans supportive only if it was an incredibly cheap bargain, well below $2,000 per year. Lucky birds.
Such examples do not reflect a stupid citizenry. They merely reveal that polls can easily overreach the public’s capacity to resolve complex problems. When the issue is familiar and close-at-hand—for example, the use of racial quotas in schools, fear of crime, public morality—people can respond with a degree of thoughtfulness. But when the poll ventures into far more complicated, distant, and unusual matters, responses become less meaningful. What is most unfortunate is that polls, by their very design, do not permit a differentiation of the two. By their format, wording, and vagueness, questionnaires disguise irresponsible, thoughtless responses. Discovering reality is replaced by shadowmetrics. People can be irresponsible and indulge their fantasies. The same citizen who refuses a beggar a dollar gladly confesses to a pollster a willingness to spend millions abolishing poverty. The latter response is the “real” one. Unfortunately, since polling is a commercial enterprise, there are no industry incentives to doubt the marketed product’s value.
In the end, perhaps some deep psychological need governs our obsession with polling: our compulsive craving for self-understanding whets our appetites for the equivalent of scientific junk food. But when confronted with junk food, the prudent will either consume in moderation or exercise restraint. When people possess firsthand familiarity with issues, polls may reveal something of value; they will render unto the public what is within the proper realm of public competence. Otherwise, be careful.