Fun with Open-Ended Responses

A few weeks ago, we talked about some of the trade-offs that come with including open-ended questions in polling instruments. On the one hand, they tend to elicit a lower response rate than closed-ended questions and can make data analysis more complicated and subjective. On the other hand, they can produce a richer and more accurate understanding of public opinion by giving respondents an opportunity to voice opinions and rationales that would not have been included among the response alternatives to closed-ended questions.

This Gallup report on a USAToday/Gallup poll conducted last month illustrates how open-ended questions can be put to good use. As the report describes, one item in the poll asked respondents how they would "describe the federal government in one word or phrase." Perhaps unsurprisingly, most offered a negative response.

That nearly three-quarters of Americans think poorly of the federal government probably could have been discovered just as easily with a closed-ended question (e.g. "In general, would you describe your view of the federal government as positive, negative, or neutral?"). What the open-ended responses give us is a much richer sense of why and how Americans by and large view the federal government in a negative light, which you can experience by perusing this pdf file, which contains a 23-page table of respondents' verbatim answers to this prompt.

The report also provides this "word cloud" visualization of the responses, which uses different size fonts to convey how frequently different words and phrases were used:


Chances are, a closed-ended question designed to find out why Americans view the federal government in a negative light would have included several of these descriptors in the response alternatives, but couldn't possibly have accommodated the full range, many of which might not have even occurred to the pollsters who developed the questionnaire.