Model Politics

I just added a new link to the "Public Opinion Resources" list in the right sidebar: Model Politics is a new blog where a group of political scientists write about public opinion on current political events and issues. Recent posts there have ranged pretty widely, covering topics like President Obama's popularity, racial prejudice, support for taxation, Americans' sense of how many people make over $250,000, and whether people who are openly gay should be allowed to teach in public schools.

Grammar Matters

While it isn't about polling per se, this Washington Post article illustrates how "loaded" terminology can affect the way people respond to prompts for their opinions about political and policy matters.

Consider the example of how people respond differently to different terms used to denote a tax on inheritance:

An "estate tax" is more likely to garner popular support than a "death tax," even though they're the same thing.

A pollster interested in determining the public's views on such taxes would likely get very different results depending on which phrase s/he decides to use in a questionnaire.

The article's main point, though, is that it's not just word choice that matters -- it turns out people's responses are affected by more subtle grammatical structures as well. Perhaps not a lot of people could explain the differences among the various past tenses, but the article suggests that they react differently to prompts depending on which conjugation is used by discussing the results of an experiment performed on undergraduate student subjects:

Half saw "Last year, Mark was having an affair with his assistant and was taking hush money from a prominent constituent." The other half saw this: "Last year, Mark had an affair with his assistant and took hush money from a prominent constituent." The difference is one of grammatical aspect: "was having" and "was taking" are known as the imperfect aspect, meaning an event may be continuing. But "had" and "took" are known as the perfect aspect, meaning the event is bounded in time.

Although the differences may seem subtle, they had a strong impact on the readers. More than three-quarters of students who read the imperfect aspect phrases said they were confident that Johnson would not be reelected, whereas only about half who read the perfect aspect phrases felt this way.

In short, it's not just which words you use that can affect poll results -- how you incorporate even the most well-chosen words into sentence stuctures can have a big impact as well.

Do People View Government as a Threat?

A couple of days ago, Gallup released this report about Americans' perception of the federal government as an immediate threat to citizens' rights and freedoms.

The report focuses especially on how those perceptions have changed over time. Taking the U.S. population as a whole, without making any subgroup distinctions, it doesn't look like much has changed. In 2006, 44% of all respondents said "yes" -- that is, that they did perceive the federal government to be an immediate threat to citizens' rights and freedoms. In 2010, that number has gone up by just 2 points, to 46%.

Breaking the population down into partisan subgroups presents a very different picture, as suggested by this graph:
Instead of practically no change, as suggested by the undifferentiated figures, we see that Republicans and Democrats have completely switched sides. In 2006, only 21% of Republicans felt that the federal government was an immediate threat to citizens; now more than 65% do; in contrast, the number of Democrats who feel the same way has dropped from more than 55% in 2006 to 21% in 2010.

Finally, to piggyback on my last post, the Gallup report also presents this table, which presents the results of an open-ended follow-up question that was asked of those 46% of respondents who said that they do perceive the federal government to be an immediate threat to citizens' rights and freedoms:

As you can see, the responses included an array of fairly general statements about ideology and overall approaches to government, as well as some responses that zeroed in on relatively narrow, single-issue concerns -- a range that probably would not have been captured with a closed-ended question.

Fun with Open-Ended Responses

A few weeks ago, we talked about some of the trade-offs that come with including open-ended questions in polling instruments. On the one hand, they tend to elicit a lower response rate than closed-ended questions and can make data analysis more complicated and subjective. On the other hand, they can produce a richer and more accurate understanding of public opinion by giving respondents an opportunity to voice opinions and rationales that would not have been included among the response alternatives to closed-ended questions.

This Gallup report on a USAToday/Gallup poll conducted last month illustrates how open-ended questions can be put to good use. As the report describes, one item in the poll asked respondents how they would "describe the federal government in one word or phrase." Perhaps unsurprisingly, most offered a negative response.

That nearly three-quarters of Americans think poorly of the federal government probably could have been discovered just as easily with a closed-ended question (e.g. "In general, would you describe your view of the federal government as positive, negative, or neutral?"). What the open-ended responses give us is a much richer sense of why and how Americans by and large view the federal government in a negative light, which you can experience by perusing this pdf file, which contains a 23-page table of respondents' verbatim answers to this prompt.

The report also provides this "word cloud" visualization of the responses, which uses different size fonts to convey how frequently different words and phrases were used:


Chances are, a closed-ended question designed to find out why Americans view the federal government in a negative light would have included several of these descriptors in the response alternatives, but couldn't possibly have accommodated the full range, many of which might not have even occurred to the pollsters who developed the questionnaire.

Public Opinion on YouTube

Here's the clip we watched last week about race and gender of interviewer effects during the 2008 election season:



Other AAPOR conference interviews that may be of interested include this one, which discusses an experiment the researchers used to estimate social desirability response bias (SDRB), a concept we discussed in class a few weeks ago:



this one, which discusses the use of Second Life to recruit survey respondents:



and this one, which discusses the challenges of respresenting Hispanics well in general public opinion surveys: