Sociological Images

Dmitriy T.M. sent in a TED talk in which Ben Goldacre discusses the problems with many of the scientific findings we hear about in the media, highlighting the importance of scientific literacy and critical consumption of science reporting:

And while we’re on the topic of potentially misleading statistics, Dolores R. and Sarah E. sent in an image posted at boing boing as one of “the best set of infographics ever,” helpfully illustrating the difference between correlation and causation:

Cross-posted at Montclair SocioBlog.

Ethnographers worry that their mere presence on the scene may be influencing what people do and thus compromising the truth of their studies.  They try to minimize that impact, and most of their reports give detailed descriptions of their methods so that readers can assess whether the data might be corrupted.

Photojournalists also claim to be showing us the truth — “pictures don’t lie” — but they compunctions about influencing the people in their photos.  Here for example is a photo taken in Israel by Italian photographer Ruben Salvadori.  (This is a screen grab of a video, hence the subtitles.)

The defiant Palestinian youth, the flames of the roadblock — it’s all very dramatic.  But it is far from spontaneous.  Here’s the photo from another point of view:

Salvadori studied anthropology, and he is well aware that observers influence what they observe.  But editors want “good” photos, not good ethnography.  So observer influence is an asset, not a problem.

If you point a tiny camera at somebody, what is he going to do?  Most likely, he’s going to smile or do something.  Now imagine this enlarged with a group of photographers. That show up with helmets, gas masks, and at least two large cameras each, and they come there to take photos of what you do.  So you’re not going to sit there twiddling your thumbs.

No, the youths don’t twiddle their thumbs, not with the photogs on the scene.  Instead, they burn a flag.

Their relationship is symbiotic.  The photogs want dramatic images, the insurgent youths want publicity.  Of course, even with the Palestinians youths and the Israeli soldiers, when the action gets real, nobody is thinking about how they’ll look in a photo.

—————————

(The full 8-minute video of Salvadori talking about photography in the combat zone was posted at PetaPixel back in October, though I didn’t hear about it until recently.)

Cross-posted at Family Inequality.

There’s an interesting example of how to interpret scientific results — and draw policy implications from them — from the world of birth practices and safety.

The subject of the debate is a major new study from the British Medical Journal. The study followed more than 60,000 women in England with uncomplicated pregnancies, excluding those who had planned caesarean sections and caesarean sections before the start of labor. They compared the number of bad outcomes — from death to broken clavicles – for women depending on where they had their births.

One comparison stands out in the results. From the abstract: “For nulliparous women [those having their first birth], the odds of the primary outcome [that is, any of the negative events] were higher for planned home births” than among those planned for delivery in obstetric units. That is, the home births had higher rates of negative events. The difference is large. Here’s a figure to illustrate:

The error bars show 95% confidence intervals, so you can see the difference between home births and obstetric-unit births is statistically significant at that level. These are the raw comparisons, but the home-versus-obstetric comparison was unchanged when the analysts controlled for age, ethnicity, understanding of English, marital or partner status, body mass index, “deprivation score,” previous pregnancies, and weeks of gestation. Further, by restricting the comparison to uncomplicated pregnancies and excluded all but last-minute c-sections, it seems to be a very strong result.

But what to make of it?

In their conclusion, the authors write:

Our results support a policy of offering healthy nulliparous and multiparous women with low risk pregnancies a choice of birth setting. Adverse perinatal outcomes are uncommon in all settings, while interventions during labour and birth are much less common for births planned in non-obstetric unit settings. For nulliparous women, there is some evidence that planning birth at home is associated with a higher risk of an adverse perinatal outcome.

But in what way do the results “support a policy”? The “higher risks” they found for planned home births are still “uncommon,” by comparison, with those in poor countries, for example. But the home birth risk is 2.7-times greater.

The Skeptical OB, who is a reliable proponent of modern medical births, titled her post, “It’s official: homebirth increases the risk of death.” She added some tables from the supplemental material, showing the type of negative events and conditions that occurred. Her conclusion:

“In other words, any way you choose to look at it, no matter how carefully you slice and dice the data, there is simply no getting around the fact that homebirth increases the risk of perinatal death and brain damage.”

I guess the policy options might include include whether home births should be encouraged, more regulated, covered by public and/or private health insurance, banned, penalized or (further) stigmatized.

Home birth seems safer than letting children ride around unrestrained in the back of pickup trucks, which is legal in North Carolina – as long as they’re engaged in agricultural labor. On the other hand, we have helmet laws for kids on bicycles in many places. And if a child is injured in either situation, hopefully an ambulance would take them to the hospital even if the accident were preventable.

In other words, I don’t think policy questions can be resolved by a comparison of risks, however rigorous.

Thanks to Flowing Data, we can now enjoy this:

Cross-posted at Family Inequality.

Poverty is usually described as a status, as there are people below and above the poverty line. We need to do more to capture and represent the experience of poverty.

There are ways this can be done even in a single survey question, such as this one: ”During the past 12 months, was there any time when you needed prescription medicine but didn’t get it because you couldn’t afford it?” Below are the percentages answering affirmatively, by official poverty-line status.

Percentage of Adults Aged 18-64 Who Did Not Get Needed Prescription Drugs Because of Cost, by Poverty Status (National Health Interview Survey, 1999-2010)

This is not the same as not having any of the prescription drugs you need. What it indicates is economic insecurity rather than deprivation per se, a more nuanced measure than simply being above or below (some percentage of) the poverty line.

Cross-posted at Family Inequality.

Things that make you say… “peer review”?

This is the time of year when I expect to read inflated or distorted claims about the benefits of marriage and religion from the National Marriage Project. So I was happy to see the new State of Our Unions report put out by W. Bradford Wilcox’s outfit. My first reading led to a few questions.

First: When they do the “Survey of Marital Generosity” — the privately funded, self-described nationally-representative sample of 18-46-year old Americans, which is the source of this and several other reports, none of them published in any peer-reviewed source I can find — do they introduce themselves to the respondents by saying, “Hello, I’m calling from the Survey of Marital Generosity, and I’d like to ask you a few questions about…” If this were the kind of thing subject to peer review, and I were a reviewer, I would wonder if the respondents were told the name of the survey.

Second: When you see oddly repetitive numbers in a figure showing regression results, don’t you just wonder what’s going on?

Here’s what jumped out at me:

If a student came to my office with these results and said, “Wow, look at the big effect of joint religious practice on marital success,” I’d say, “Those numbers are probably wrong.” I can’t swear they didn’t get exactly the same values for everyone except those couples who both attend religious services regularly — 50 50 50, 13 13 13 , 50 50 50, 21 21 21 — in a regression that adjusts for age, education, income, and race/ethnicity, but that’s only because I don’t swear.*

Of course, the results are beside the point in this report, since the conclusions are so far from the data anyway. From this figure, for example, they conclude:

In all likelihood,  the experience of sharing regular religious attendance — that  is, of enjoying shared rituals that endow one’s marriage with transcendent significance and the support of a community  of family and friends who take one’s marriage seriously— is a solidifying force for marriage in a world in which family life is  increasingly fragile.

OK.

Anyway, whatever presumed error led to that figure seems to reoccur in the next one, at least for happiness:

Just to be clear with the grad student example, I wouldn’t assume the grad student was deliberately cooking the data to get a favorable result, because I like to assume the best about people. Also, people who cook data tend to produce a little more random-looking variation. Also, I would expect the student not to just publish the result online before anyone with a little more expertise had a look at it.

Evidence of a pattern of error is also found in this figure, which also shows predicted percentages that are “very happy,” when age, education, income and race/ethnicity are controlled.

Their point here is that people with lots of kids are happy (which they reasonably suggest may result from a selection effect). But my concern is that the predicted percentages are all between 13% and 26%, while the figures above show percentages that are all between 50% and 76%.

So, in addition to the previous figures probably being wrong, I don’t think this one can be right unless they are wrong. (And I would include “mislabeled” under the heading “wrong,” since the thing is already published and trumpeted to the credulous media.)

Publishing apparently-shoddy work like this without peer review is worse when it happens to support your obvious political agenda. One is tempted to believe that if the error-prone research assistant had produced figures that didn’t conform to the script, someone higher up might have sent the tables back for some error checking. I don’t want to believe that, though, because I like to assume the best about people.

* Just kidding. I do swear.

Cross-posted at Montclair SocioBlog.

The best way to lie with statistics, says Andrew Gelman, is just lie.  This graph from Fox news is a visual version of that.  It’s published at Flowingdata.com via Media Matters.

The numbers are correct, but the Foxy graphmongers are making up the Y-axis as they go along.  The 8.6% of November is higher than than 8.8%, 8.9%, and maybe even the 9.0% of the first three months of the year.

Or maybe it’s an optical illusion.

[HT:  Max Livingston]