methods/use of data

In statistics, a little star next to a coefficient generally means that the result is statistically significant at the p<.05 level. In English, this means that there is only a 1 in 20 chance that the finding just popped up by pure random chance. In sociology, that’s generally considered good enough to conclude that the finding is “real.”

If one investigates a lot of relationships, however, this way of deciding which ones to claim as real has an obvious pitfall.  If you look at 20 possible but false relationships, chances are that one of them will be statistically significant by chance alone. Do enough fishing in a dead lake, in other words, and you’ll inevitably pull up some garbage.

Thanks xkcd, for making this funny.

Lisa Wade, PhD is an Associate Professor at Tulane University. She is the author of American Hookup, a book about college sexual culture; a textbook about gender; and a forthcoming introductory text: Terrible Magnificent Sociology. You can follow her on Twitter and Instagram.

At Junk Charts, Kaiser Fung drew my attention to a graph released by Reuters.  It is so deeply misleading that I loathe to expose your eyeballs to it.  So, I offer you this:

The original figure is on the left.  It counts the number of gun deaths in Florida.  A line rises, bounces a little, reaches a 2nd highest peak labeled “2005, Florida enacted its ‘Stand Your Ground’ law,” and falls precipitously.

What do you see?

Most people see a huge fall-off in the number of gun deaths after Stand Your Ground was passed.  But that’s not what the graph shows.  A quick look at the vertical axis reveals that the gun deaths are counted from top (0) to bottom (800).  The highest peaks are the fewest gun deaths and the lowest ones are the most.  A rise in the line, in other words, reveals a reduction in gun deaths.  The graph on the right — flipped both horizontally and vertically — is more intuitive to most: a rising line reflects a rise in the number of gun deaths and a dropping a drop.

The proper conclusion, then, is that gun deaths skyrocketed after Stand Your Ground was enacted.

This example is a great reminder that we bring our own assumptions to our reading of any illustration of data.  The original graph may have broken convention, making the intuitive read of the image incorrect, but the data is, presumably, sound.  It’s our responsibility, then, to always do our due diligence in absorbing information.  The alternative is to be duped.

Cross-posted at Pacific Standard.

Lisa Wade, PhD is an Associate Professor at Tulane University. She is the author of American Hookup, a book about college sexual culture; a textbook about gender; and a forthcoming introductory text: Terrible Magnificent Sociology. You can follow her on Twitter and Instagram.

Flashback Friday.

In a great book, The Averaged American, sociologist Sarah Igo uses case studies to tell the intellectual history of statistics, polling, and sampling. The premise is fascinating:  Today we’re bombarded with statistics about the U.S. population, but this is a new development.  Before the science developed, the concept was elusive and the knowledge was impossible. In other words, before statistics, there was no “average American.”

There are lots of fascinating insights in her book, but a post by Byron York brought one in particular to mind.  Here’s a screenshot of his opening lines (emphasis added by Jay Livingston):

The implication here is, of course, that Black Americans aren’t “real” Americans and that including them in opinion poll data is literally skewing the results.

Scientists designed the famous Middletown study with exactly this mentality.  Trying to determine who the average American was, scientists excluded Black Americans out of hand.  Of course, that was in the 1920s and ’30s.  How wild to see the same mentality in the 2000s.

Originally posted in 2009.

Lisa Wade, PhD is an Associate Professor at Tulane University. She is the author of American Hookup, a book about college sexual culture; a textbook about gender; and a forthcoming introductory text: Terrible Magnificent Sociology. You can follow her on Twitter and Instagram.

First, there were the accolades. More than 100 instances of street harassment in a two minute video, testifying powerfully to the routine invasion of women’s lives by male strangers.

Then, there was the criticism. How is it, people asked, that the majority of the men are black? They argued: this video isn’t an indictment of men, it’s an indictment of black men.

Now, we’ve reached the third stage: lessons for research methods classes.

Our instructor is sociologist Zeynep Tufekci, writing at The Message. Our competing hypotheses are three:

1. Black men really do catcall more than other kinds of men.

2. The people who made this video are unconsciously or consciously racist, editing out men of other races.

3. The study was badly designed.

As Tufekci points out, any one of these could account for why so many of the catcallers were black. Likewise, all three could be at play at once.

Enter, the data wrangler: Chris Moore at Mass Appeal.

Moore and his colleagues looked for landmarks in the video in order to place every instance of harassment on the map of New York City. According to their analysis, over half of the harassment occurs on just one street — 125th — in Harlem.

Did the time the producers spent in Harlem involve denser rates of harassment, supporting hypothesis #1. Did they spend an extra amount of time in Harlem because they have something against black men? That’d be hypothesis #2. Or is it hypothesis #3: they were thoughtless about their decisions as to where they would do their filming.

Honestly, it’s hard to say without more data, such as knowing how much time they spent in each neighborhood and in neighborhoods not represented in the video. But if it’s true that they failed to sample the streets of New York City in any meaningful way — and I suspect it is — then hypothesis #3 explains at least some of why black men are over-represented.

And that fact should motivate us all to do our methods right. If we don’t, we may end up offering accidental and fallacious support to ideas that we loathe.

Cross-posted at Pacific Standard.

Lisa Wade, PhD is an Associate Professor at Tulane University. She is the author of American Hookup, a book about college sexual culture; a textbook about gender; and a forthcoming introductory text: Terrible Magnificent Sociology. You can follow her on Twitter and Instagram.

Great comic by xkcd.

Originally posted in 2009.

Lisa Wade, PhD is an Associate Professor at Tulane University. She is the author of American Hookup, a book about college sexual culture; a textbook about gender; and a forthcoming introductory text: Terrible Magnificent Sociology. You can follow her on Twitter and Instagram.

Flashback Friday.

The Tuskegee Syphilis Experiment is one of the most famous examples of unethical research. The study, funded by the federal government from 1932-1972, looked at the effects of untreated syphilis. In order to do this, a number of Black men in Alabama who had syphilis were misinformed about their illness. They were told they had “bad blood” (which was sometimes a euphemism for syphilis, though not always) and that the government was offering special free treatments for the condition. Here is an example of a letter sent out to the men to recruit them for more examinations:

The “special free treatment” was, in fact, nothing of the sort. The researchers conducted various examinations, including spinal taps, not to treat syphilis but just to see what its effects were. In fact, by the 1950s it was well established that a shot of penicillin would fully cure early-stage syphilis. Not only were the men not offered this life-saving treatment, the researchers conspired to be sure they didn’t find out about it, getting local doctors to agree that if any of the study subjects came in they wouldn’t tell them they had syphilis or that a cure was available.

The abusive nature of this study is obvious (letting men die slow deaths that could have been easily prevented, just for the sake of scientific curiosity) and shows the ways that racism can influence researchers’ evaluations of what is acceptable risk and whose lives matter. The Tuskegee experiment was a major cause for the emergence of human subjects protection requirements and oversight of federally-funded research once the study was exposed in the early 1970s. Some scholars argue that knowledge of the Tuskegee study increased African Americans’ distrust of the medical community, a suspicion that lingers to this day.

In 1997 President Clinton officially apologized for the experiment.

Originally posted in 2009.

Gwen Sharp is an associate professor of sociology at Nevada State College. You can follow her on Twitter at @gwensharpnv.

What should we make of changes in fashion? Are they the visible outward expression of new ways of thinking? Or do fashions themselves influence our sentiments and ideas? Or are fashions merely superficial and without any deeper meaning except that of being fashionable?

It’s summer, and once again magazines and newspapers are reporting on beachwear trends in France, proclaiming “the end of topless.” They said the same thing five years ago.

As in 2009, no systematic observers were actually counting the covered and uncovered chests on the beach. Instead, we are again relying on surveys – what people say they do, or have done, or would do.  Elle cites an Ipsos survey: “In 2013, 93% of French women say that they wear a top, and 35% find it ‘unthinkable’ to uncover their chest in public.”

Let’s assume that people’s impressions and the media stories are accurate and that fewer French women are going topless. Some of stories mention health concerns, but most are hunting for grander meanings. The Elle cover suggests that the change encompasses issues like liberty, intimacy, and modesty.  Marie-Claire says,

Et en dehors de cette question sanitaire, comment expliquer le recul du monokini : nouvelle pudeur ou perte des convictions féministes du départ ?

But aside from the question of health, how to explain the retreat from the monokini: a new modesty or a loss of the original feminist convictions? [my translation, perhaps inaccurate]

The assumption here is that is that ideas influence swimwear choices.  Women these days have different attitudes, feelings, and ideologies, so they choose apparel more compatible with those ideas.  The notion certainly fits with the evidence on cultural differences, such as those between France and the U.S.

Americans are much more likely to feel uncomfortable at a topless beach. But they are also much less likely to have been to one. (Northern Europeans – those from the Scandinavian countries and Germany – are even more likely than the French to have gone topless.) (Data are from a 2013 Harris survey done for Expedia.)

This second graph could also support the other way of thinking about the relation between fashion and ideas: exposing your body changes how you think about bodies.  If people take off their clothes, they’ll become more comfortable with nudity. That is, whatever a woman’s original motivation, once she did try going topless, she would develop ideas that made sense of the experiences, especially since the body already carries such a heavy symbolism. She would not have to invent these topless-is-OK ideas all by herself. They would be available in the conversations of others. So unless her experiences were negative, these new ideas would add to and reinforce the thoughts that led to the original behavior.

This process is much like the general scenario Howie Becker outlines for deviance.

Instead of deviant motives leading to deviant behavior, it is the other way around; the deviant behavior in time produces the deviant motivation.  Vague impulses and desires … probably most frequently a curiosity … are transformed into definite patterns of action through social interpretation of a physical experience. [Outsiders, p. 42]

With swimwear, another motive besides “vague impulses” comes into play:  fashion –  the pressure to wear something that’s within the range of what others on the beach are wearing.

Becker was writing about deviance.  But when the behavior is not illegal and not all that deviant, when you can see lots of people doing it in public, the supportive interpretations will be easy to come by.  In any case, it seems that the learned motivation stays learned.  The fin-du-topless stories,  both in 2009 and 2014, suggest that the change is one of generations rather than a change in attitudes.  Older women have largely kept their ideas about toplessness. And if it’s true that French women don’t get fat, maybe they’ve even kept their old monokinis.  It’s the younger French women who are keeping their tops on. But I would be reluctant to leap from that one fashion trend to a picture of an entire generation as more sexually conservative.

Jay Livingston is the chair of the Sociology Department at Montclair State University. You can follow him at Montclair SocioBlog or on Twitter.

Great comic by David Malki at Wondermark.  H/t to @annettecboehm.

Lisa Wade, PhD is an Associate Professor at Tulane University. She is the author of American Hookup, a book about college sexual culture; a textbook about gender; and a forthcoming introductory text: Terrible Magnificent Sociology. You can follow her on Twitter and Instagram.