{"id":42866,"date":"2011-12-15T12:54:08","date_gmt":"2011-12-15T17:54:08","guid":{"rendered":"http:\/\/thesocietypages.org\/socimages\/?p=42866"},"modified":"2011-12-13T17:59:58","modified_gmt":"2011-12-13T22:59:58","slug":"church-saves-marriage-and-produces-curious-coefficients","status":"publish","type":"post","link":"https:\/\/thesocietypages.org\/socimages\/2011\/12\/15\/church-saves-marriage-and-produces-curious-coefficients\/","title":{"rendered":"Church Saves Marriage… and Produces Curious Coefficients"},"content":{"rendered":"

Cross-posted at Family Inequality<\/a>.<\/em><\/p>\n

Things that make you say\u2026 \u201cpeer review\u201d?<\/p>\n

This is the\u00a0time of year<\/a> when I expect to read\u00a0inflated or distorted claims<\/a> about the benefits of\u00a0marriage and religion<\/a> from the\u00a0National Marriage Project.<\/a> So I was happy to see the new\u00a0State of Our Unions<\/a> report put out by W. Bradford Wilcox\u2019s outfit. My first reading led to a few questions.<\/p>\n

First: When they do the \u201cSurvey of Marital Generosity\u201d \u2014 the privately funded, self-described nationally-representative sample of 18-46-year old Americans, which is the source of this and several other reports, none of them published in any peer-reviewed source I can find \u2014 do they introduce themselves to the respondents by saying, \u201cHello, I\u2019m calling from the Survey of Marital Generosity, and I\u2019d like to ask you a few questions about\u2026\u201d If this were the kind of thing subject to peer review, and I were a reviewer, I would wonder if the respondents were told the name of the survey.<\/p>\n

Second: When you see oddly repetitive numbers in a figure showing regression results, don\u2019t you just wonder what\u2019s going on?<\/p>\n

Here\u2019s what jumped out at me:<\/p>\n

\"\"<\/a><\/p>\n

If a student came to my office with these results and said, \u201cWow, look at the big effect of joint religious practice on marital success,\u201d I\u2019d say, \u201cThose numbers are probably wrong.\u201d I can\u2019t\u00a0swear<\/em> they didn\u2019t get exactly the same values for everyone except those couples who both attend religious services regularly \u2014 50 50 50, 13 13 13 , 50 50 50, 21 21 21 \u2014 in a regression that adjusts for age, education, income, and race\/ethnicity, but that\u2019s only because I don\u2019t swear.*<\/p>\n

Of course, the results are beside the point in this report, since the conclusions are so far from the data anyway. From this figure, for example, they conclude:<\/p>\n

In all likelihood,\u00a0 the experience of sharing regular religious attendance \u2014 that\u00a0 is, of enjoying shared rituals that endow one\u2019s marriage with transcendent significance and the support of a community\u00a0 of family and friends who take one\u2019s marriage seriously\u2014 is a solidifying force for marriage in a world in which family life is\u00a0 increasingly fragile.<\/p><\/blockquote>\n

OK.<\/p>\n

Anyway, whatever presumed error led to that figure seems to reoccur in the next one, at least for happiness:<\/p>\n

\"\"<\/a><\/p>\n

Just to be clear with the grad student example, I wouldn\u2019t assume the grad student was deliberately cooking the data to get a favorable result, because I like to assume the best about people. Also, people who cook data tend to produce a little more\u00a0random-looking variation<\/a>. Also, I would expect the student not to just publish the result online before anyone with a little more expertise had a look at it.<\/p>\n

Evidence of a pattern of error is also found in this figure, which also shows predicted percentages that are \u201cvery happy,\u201d when age, education, income and race\/ethnicity are controlled.<\/p>\n

\"\"<\/a>Their point here is that people with lots of kids are happy (which they reasonably suggest may result from a selection effect). But my concern is that the predicted percentages are all between 13% and 26%, while the figures above show percentages that are all between 50% and 76%.<\/p>\n

So, in addition to the previous figures probably being wrong, I don\u2019t think this one can be right unless they are wrong. (And I would include \u201cmislabeled\u201d under the heading \u201cwrong,\u201d since the thing is already published and\u00a0trumpeted to the credulous media<\/a>.)<\/p>\n

Publishing apparently-shoddy work like this without peer review is worse when it happens to support your obvious political agenda. One is tempted to believe that if the error-prone research assistant had produced figures that didn\u2019t conform to the script, someone higher up might have sent the tables back for some error checking. I don\u2019t want to believe that, though, because I like to assume the best about people.<\/p>\n

* Just kidding. I do swear.<\/p>\n","protected":false},"excerpt":{"rendered":"

Cross-posted at Family Inequality. Things that make you say\u2026 \u201cpeer review\u201d? This is the\u00a0time of year when I expect to read\u00a0inflated or distorted claims about the benefits of\u00a0marriage and religion from the\u00a0National Marriage Project. So I was happy to see the new\u00a0State of Our Unions report put out by W. Bradford Wilcox\u2019s outfit. My first […]<\/p>\n","protected":false},"author":287,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[223,272,274,42],"class_list":["post-42866","post","type-post","status-publish","format-standard","hentry","category-uncategorized","tag-childrenyouth","tag-marriagefamily","tag-methodsuse-of-data","tag-religion"],"jetpack_featured_media_url":"","_links":{"self":[{"href":"https:\/\/thesocietypages.org\/socimages\/wp-json\/wp\/v2\/posts\/42866","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/thesocietypages.org\/socimages\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/thesocietypages.org\/socimages\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/thesocietypages.org\/socimages\/wp-json\/wp\/v2\/users\/287"}],"replies":[{"embeddable":true,"href":"https:\/\/thesocietypages.org\/socimages\/wp-json\/wp\/v2\/comments?post=42866"}],"version-history":[{"count":7,"href":"https:\/\/thesocietypages.org\/socimages\/wp-json\/wp\/v2\/posts\/42866\/revisions"}],"predecessor-version":[{"id":42884,"href":"https:\/\/thesocietypages.org\/socimages\/wp-json\/wp\/v2\/posts\/42866\/revisions\/42884"}],"wp:attachment":[{"href":"https:\/\/thesocietypages.org\/socimages\/wp-json\/wp\/v2\/media?parent=42866"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/thesocietypages.org\/socimages\/wp-json\/wp\/v2\/categories?post=42866"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/thesocietypages.org\/socimages\/wp-json\/wp\/v2\/tags?post=42866"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}