This clip from The Daily Show nicely illustrates how ridiculous and utterly meaningless the statistics we encounter can be:
This probably goes without saying, but there are multiple problems here:
1. Viewers of these shows are a self-selected group who are quite likely watching because they agree with the hosts to some degree, so it wouldn’t be surprising they’d agree with the hosts’ views.
2. Viewers who care enough to text are an even more unusual group, likely to be those who feel most passionately about an issue.
3. Only those people watching the show and are able to text right then are able to vote.
4. The wording of the questions is clearly intended to lead to a particular answer, using leading phrases like “are you outraged,” which responsible social scientists would never use–any question that uses something along the lines of “don’t you agree” or “wouldn’t you say that” makes it more likely the respondents will, indeed, agree with the point.
5. The hosts actively cajole viewers to give a specific answer if they aren’t getting as many of that answer as they wanted.
Of course, the hosts aren’t trying to present factual, useful information and almost certainly know very well that they’re manipulating questions to get results that will appear to overwhelmingly support their position. But we’re inundated with “statistics” such as these every day that are completely meaningless, but many many people don’t know how to evaluate them. This little clip shows some of the things a person should look for as an indication that a number was created to support a particular viewpoint and should be viewed with extreme skepticism, if not dismissed altogether.
Gwen Sharp is an associate professor of sociology at Nevada State College. You can follow her on Twitter at @gwensharpnv.
Comments 7
rachel — August 19, 2009
excellent. I loved the clip.
Posts about Jon Stewart as of August 19, 2009 » The Daily Parr — August 19, 2009
[...] our count well over 1/3 of the allotted funds have been spent and GM hopes to grab some of the Meaningless Statistics from Media Outlets - thesocietypages.org 08/19/2009 This clip from The Daily Show nicely illustrates how ridiculous and [...]
Lance — August 19, 2009
What was perhaps even better was the Moment of Zen, in which Bill O'Reilly explained that the results of one of his polls were so close to 50/50 because a website had encouraged left-wing loonies to vote in it. It really just spoke for itself.
Dmitriy — August 19, 2009
I really wish this website would have a poll so we could vote whether we like/dislike posts like these.
Emily — August 19, 2009
A 6th possibility is that they stop counting after a few texts. One of the polls yielded a result of 100% to 0%. The first text might have been a "yes," and they just stopped letting texts go through after that.
Annoyed — August 20, 2009
It's depressing because people don't know how to evaluate and start to mistrust or disregard all statistics and all scientific studies. We have a situation where it has become really easy to discount and point out all the limitations with studies we don't agree with and hold up as "science" the studies we do agree with.
The NRA and Texas Twang « Suburban Kitsch — September 2, 2009
[...] for clarification. The NRA clearly worded the question to lead to their desired answer (see Meaningless Statistics from Sociological Images, specifically #4). Now really, REALLY… who the hell could POSSIBLY [...]