a 2013 CNLG workshop
a 2013 CNLG workshop

My spring break involved an intense research trip to Rwanda with Hollie Nyseth Brehm, but we reserved the last day for something fun:  a methodology workshop for researchers at The National Commission For The Fight Against Genocide. We gave a whirlwind presentation from 9-12, then held office hours to address specific projects. I worried we wouldn’t get much traffic, but the researchers had so many sharp and important questions that they kept us hopping all afternoon.

With so much recent hand-wringing over the relevance and impact of social research, it was inspiring to connect with researchers squarely addressing problems of unquestioned importance —  understanding a genocide that took a million lives, investigating and ameliorating its effects on survivors,  and working to prevent its recurrence within and outside Rwanda. So I began by asking, “Whose research is more important than yours?” and then stressed the central role of design and methodology in getting it right.

The researchers were super-smart and accomplished but came to us with varying levels of methods training. We therefore emphasized building a strong foundation: posing tight research questions, theory and conceptualization, levels of analysis, data and valid measures, sampling difficult-to-study populations, criteria of causality, research ethics and positionality, publishing and dissemination, and specific issues in interviewing, comparative and historical analysis, and univariate, bivariate, and multivariate statistical analysis. That’s a lot for a semester, let alone a half-day workshop. Throughout, we discussed the importance of humility in doing good research — listening, learning, and keeping an open mind. As is often the case in teaching, we were humbled by the collective expertise and dedication among the researchers. On the subject of “sensitive interview questions,” for example, I turned the microphone over to a participant who had just finished a set of interviews that were an order of magnitude more sensitive than any I’d conducted in my research career.

We can’t say whether or how our little methods session might affect the Center’s research, but Hollie and I certainly gained much from the experience. Yes, we “know stuff” as social scientists, but we have also developed a wonderfully useful set of tools for acquiring this knowledge. And once researchers understand the basic idea of something like multistage cluster sampling, they can often make thoughtful design choices that yield better research with fewer resources. It may seem too obvious to mention, but many of our methods discussions within the field seem devoted to “tweaking and critiquing” more esoteric designs and models. So it is both refreshing and empowering to share some of the methodological foundations of our field with such sharp and motivated researchers.

strainer
Photo via Marina Noordegraaf flickr.com CC.

Social media feeds are like carnival money booths: we snatch away greedily as the links swirl past, but we’re rarely enriched by the experience. In the rush to process so much so quickly, we’ve become lousy filters for one another – recommending “great articles” that ain’t so great by social science standards.

Many rapidly-circulating stories offer grand assertions but paltry evidence about the social world. It seems silly to direct much intellectual horsepower at every li’l item whooshing past (why, that Upworthy post needs an interrupted time-series design!). So people just hit the “thumbs up” button if they like the sentiment and send it down the line. Passing along such blurbs can seem like a modern equivalent to the kindly/nosy relative who sent us Dear Abby clippings in the newsprint era. Yet there’s a danger to indiscriminate recommendations that can subvert our authority as experts. In my case, I’ve developed a set of policy preferences on crime and economic issues, which I adjust in response to new evidence. If I start endorsing weak studies just because they affirm my preferences or prejudices, then I’d rightly be considered a hack.

As conservatives like to remind progressives — from the comfort of their thin-paned glass houses — there’s a big honking gap between the truth about the world and the truth we’d like to believe about the world. Accordingly, there’s a big honking gap between a “great study” and a “great sentiment” that neatly aligns with our views. And, unlike your kindly/nosy relative, good social scientists have a real responsibility to evaluate the quality of the evidence we cite – especially when we claim to be experts on a matter.

Sometimes we forget that social science provides mighty tools and deep training in evaluating evidence. For example, any good sociologist should have a pretty good sense of whether a given sample is likely to be representative; whether a design is best suited for making causal, descriptive, or interpretive claims; whether to gather data from individuals, groups, or nations in making such claims; and, how to make sense of complex processes that unfold dynamically across all these levels. But while we might closely and carefully scrutinize research methods in our professional work, we seem to get beer goggles whenever a sexy story flits past on Facebook.

When I suspect I might be playing too fast and loose with such stories, I use a three-step approach to consider the evidence:

  1. Restate the central empirical claim (e.g., raising the minimum wage reduces crime)
  2. Identify the theory and evidence cited to support that claim (e.g., a simple plot showing lower crime rates in states with higher minimum wage levels)
  3. Evaluate the design rather than the finding. Is the design so elegant and convincing that I would have believed the results had they gone the other way? Or would I have simply dismissed it as shoddy work? (e.g., a simple plot showing higher crime rates in states with higher minimum wage levels).

Depending on the direction of the wage-crime relationship, my reaction would have changed from “See! This shows I was right all along” to “Bah! These fools didn’t even control for income and poverty rates!” Of course, few of the stories flitting past can withstand the strict scrutiny of a top peer-reviewed journal article. But while I might still circulate them for descriptive or entertainment value, I’m now making fewer unqualified personal recommendations. I’d rather reserve the term “great study” for designs that are so spine-crushingly beautiful that they might actually change my mind on an issue. Researchers know that winning over skeptics is way more fun — and way more important — than preaching to the converted. At TheSocietyPages, this process always animates our board meetings, in lively debates about the research evidence that merits highlighting in our podcasts, citings, TROTS, reading list, and feature sections.

As Clay Shirky says, “It’s not information overload. It’s filter failure.” At TSP, we’ll do our best to screen for solid evidence and big ideas about the social world, in hopes that we can all grab something worthwhile from the information swirl.

socsciVery cool to see the first issue of Sociological Science, a new open-access journal for primary research articles. While I should disclose that I’m a consulting editor for the publication, I should also disclose that I can’t claim any credit for the good stuff therein:

UPDATE: Vol 1., No. 2. On the subject of new publications, we also want to show a little TSP love for Social Currentsthe impressive new journal of the Southern Sociological Society. Editors Toni Calasanto and Vinnie Roscigno just released their second issue, with some really provocative work by some of the best sociologists in the business.

I could write many more nice things about each publication, but I’ll likely be submitting my own research to Sociological Science and Social Currents — and nobody likes an apple polisher.

Picture 2

 

 

Photo via MarkShoots flickr.com CC. Click for original.
Photo via MarkShoots flickr.com CC. Click for original.

Nicholas Kristof called out professors today, saying we’ve “fostered a culture that glorifies arcane unintelligibility while disdaining impact and audience.” While the snarkmeister in me is tempted to return serve — couldn’t one say the same thing about the Times? — I actually concur with Mr. Kristof on several key points.

To paraphrase, he cites dreadful writing, a lack of political and ideological diversity, a dearth of public intellectuals, obstructionist professional associations, little social media presence, “hidden-away” journals, and a reward structure that privileges technique and abstraction over relevance, clear thinking, and broad dissemination.  In truth, we at TSP make largely the same claims in pitching our li’l project to authors, readers, and potential partners. We use different words, of course, but the whole point of TSP is to help bring social science to broader visibility and influence. This mission drives all the choices we’ve made: to stay open-access, to put our resources into a best-in-the-business professional editor and site designer, and to partner with other groups who “get” our mission and vision — like WW Norton, the Scholars Strategy Network, and Contexts magazine.

While many academics feel marginalized by mainstream media and society, Mr. Kristof points out that we’re also self-marginalizing. As a scholar, an editor, and an academic administrator, I’d agree that at least some of our injuries are self-inflicted.  For example, I was gratified when Attorney General Holder used some of my felon voting research last Monday. We’d undertaken the project with both science and policy in mind, in hopes of doing good sociology that would also encourage the sort of  national conversation now taking place. When the Times wrote a characteristically smart op-ed on Tuesday, friends asked why they linked to an old working paper rather than the polished journal article. This is likely because the article remains “hidden away” behind a paywall. I suppose they could have secured permissions from the authors, the journals, and the professional association that owns the journals, but we all tend to work on timelines that are a wee bit more protracted than the speech-on-Monday/op-ed-on-Tuesday news cycle.

While we can’t solve all of the problems of academic self-marginalization, we can at least offer Nicholas Kristof a free subscription to TheSocietyPages.org. And we’ll continue to extend the same offer to every one of the million-plus readers stopping by every month.

Picture 2

 

 

We’ve lost a true friend and remarkable colleague. Tim Ortyl was an extraordinary young sociologist and TSP grad board member. His countless friends are shocked and saddened by news that he’d passed away yesterday of natural causes due to epilepsy.  It is too early and too damn painful to post personal recollections or pictures — especially when Tim’s joy, sly wit, and vitality seem to leap from every image. But his talents and range as a sociologist are amply displayed in the publications he leaves behind, his public writing and podcasts for TSP and Contexts, and in the care and commitment with which he taught hundreds of students about statistics, gender, family, and sexuality. We mourn Tim Ortyl as a young friend with limitless potential, but we also recognize him as an accomplished and respected sociologist.

Hodges
Star-Tribune photo

I’ve long believed a graduate degree in the social sciences provides excellent preparation for elective office. We learn to critically analyze data, to abstract from individual cases to broader social processes, and to understand how both powerful institutions and grass-roots movements shape the social world. Though few U.S. sociologists have entered the fray since Pat Moynihan left the Senate, our training and experience should prepare us well for many  aspects of the political arena.

Consider today’s Star-Tribune  profile of Betsy Hodges, who is making a strong run to become mayor of Minneapolis. Ms. Hodges, who did her graduate work in sociology at Wisconsin, is characterized in the following terms:

  • “numbers-oriented and careful with her words”
  • “adept at untangling complicated financial matters”
  • “a theme of activism around social justice”
  • a concern with “people being separated from one another by things that don’t matter”
  • showing “leadership above and beyond her own stated personal views and keeping people together”

Ms. Hodges certainly possessed many of these skills and orientations before entering graduate school (though I believe that Wisconsin implanted a “numbers-oriented and careful with words” chip in all graduate students throughout the 1990s). So why don’t more of us pursue politics as a vocation? I got a glimpse of the answer when I chided a legislator for not “demonstrating courage”  on a crime policy. He said, “its a helluva lot easier to be courageous when you’re not running for reelection. Give me your university tenure and I’d demonstrate courage up the [wazoo].”  Good point, that — and all the more reason to appreciate courageous sociologist-politicians like Betsy Hodges.

 

hagan_foster13I was surprised to receive an invitation to speak at the White House this August, as part of a parental incarceration workshop sponsored by the American Bar Foundation and National Science Foundation. Though I’d written a bit on the subject and had followed the research closely for a decade, I could not claim any great expertise. Fortunately, they didn’t need me for that. They’d already assembled an impressive roster of experts to speak on topics such as demography and family dynamics, behavioral and health problems, education and exclusion, justice policy, and caring for children. My job, according to the draft agenda, was to offer “concluding comments” in the final half-hour session. Or, as John Hagan put it, “Just do what you do.”

Riiiiight. Do what I do.

Well, I couldn’t just come out and ask what I do, so I decided to do TSP. Here’s a short version of my email response:

My plan is to come in with a few minutes of my own material, but to really spend the time synthesizing and connecting across the presentations and discussants. I’ll have to do some of this on-the-fly, but I’d be delighted if you could provide the available slides in advance. If that’s not workable, that’s ok too. I’m not planning to talk for the full half-hour, but to offer some take-home points of consensus and dissensus, inviting reactions from the experts assembled. This sort of thing might be useful in a policy group (especially reprising points made in the morning sessions that get lost by afternoon). I’ll then speak briefly about points of contact with my own research. 

uggen_whitehouse_13So, after a strong kick-off by Bruce Western and a full day of panels by real experts, I took this approach at the podium. Seeing the slides in advance, it started to become clear how the research evidence fit together. The organizers had done a terrific job recruiting the experts. The experts, for their part, had made powerful new contributions to knowledge. And, throughout the day, an audience of policy leaders, practitioners, and political actors had been offering incisive commentary and questions.

As you might imagine, my notes were a hopeless mess, since I was constantly either crossing things out (when thunder was stolen) or reframing them in light of what the speakers actually said (when the best stuff wasn’t on the slides). But thinking about the talk as a TSP article, I tried to draw out five jargon-free social facts from the evidence presented — and then to connect them with the social choices and policy levers each implied.

I’m pretty sure I didn’t say anything brilliant, but I hope that I communicated something useful. Editing Contexts and TSP, I’ve learned that social scientists can sometimes be especially useful when we examine and call attention to work that is closely related but not identical to our own. And that when we take the role of reporters rather than experts, we’re pretty well positioned to identify and explain the impressive accomplishments of our colleagues.

creative commons image, courtesy Wisconsin DNR

Anyone tracking both popular and academic writing knows that media reports often miss out on the best, most directly relevant research on the story. At TSP, we’ve batted our heads against this wall for years. Yes, Citings and Sitings reports on social scientists in the press and the Reading List brings new research discoveries to light, but we’ve lacked a forum to make the connection more explicitly — to point to good work that should be cited in the news stories of the day.

So, like many of our readers, we find ourselves spitting coffee each morning and shouting, “There’s Research on That!” when the news somehow misses a perfectly on-point expert and study. [This ain’t necessarily the journo’s fault, of course. One of my favorite writers at the Times recently emailed to ask, “Can you send me your article from JournalX? Costs like 80 bucks to get.”] While we can’t take down everyone else’s paywalls (can we, Jon?), we can point readers to first-rate research on the day’s issues. And we’ll do so in a brand new TSP feature titled (what else?), There’s Research on That! From Evan Stewart of our grad board:

In our continuing quest to bring great social science to everyone, TSP is rolling out our latest blog project—There’s Research on That!, where we offer up great research from across our fields that speaks to the big events of the day. For journalists, TROT is a great place to find an interviewee or a new perspective on reporting. For general readers, TROT is a place to discuss current events and find some great book recommendations that are sure to impress at the next cocktail party. For sociologists, TROT posts can spark conversations about where our research connects with the real world. Of course, we welcome continued suggestions for pieces in the comments and on Twitter (hashtag #TROT, of course)!

We’re hoping to make this a quick-hitting and timely feature, so we’ll keep the posts short and spread the word on social media. We hope There’s Research on That informs, inspires, and provokes you — and that you’ll cite your own favorite research in the comments section.

apple_Doug88888
Image courtesy of @Doug88888 via Creative Commons

A shedload of sociologists descends on New York next week for a big annual meeting. As we scuffle for jobs and book deals or steel ourselves for presentations, the vibe can be a bit tense in the hotel lobbies. It isn’t easy to present new ideas to an audience that prides itself on the critical analysis of new ideas.

But there’s a small move you can make to improve said vibe, whether you’re a professional academic or a civilian reader who just enjoys sociological writing. Has anyone’s work inspired or influenced you? Did a writer turn a particularly memorable phrase in an article or post on TSP or elsewhere? If so, let them know about it! Send a quick note or strike up a conversation with someone whose work you’ve enjoyed and tell them so.

A good compliment is an amazing restorative – enough to sustain many of us through professional or personal rough patches. But there’s a strong professional bias against giving and receiving compliments, as sociologists take a jaundiced view of the practice. A 2012 study is titled “apple-polishers, butt-kissers, and suck-ups” and most research on compliments points to class, race, and (especially) gender disparities in ingratiation. But there’s also a grain of truth in Oscar Wilde’s admonition in Lady Windermere’s Fan: it is a great mistake to give up paying compliments, “for when we give up saying what is charming, we give up thinking what is charming.”

Compliments can be an unexpected delight — people noticing your name tag or sending an email out of the blue (especially when you’re not chairing a hiring committee). And the more obscure and left-field the compliment, the better. Kind words about a newsletter piece, a talk for a community organization, or a small contribution to a book that sold 5 copies are especially appreciated. Looking over the past year, did you find something charming or true in one piece you read? Or, perhaps, in a piece of a piece you read? If so, the author would like to hear about it.

If you’re so inclined, here are a few general characteristics and specific examples of good compliments, as distinct from simple schmoozing. The first is the most important; if you’re not feeling it, the recipient won’t either. And do try to avoid backhanded compliments (saying something positive, and then bringing the nasty).

1. Genuine

  • Good: “I was struggling with the method until I read your description in that AJR article – it was so clear! I can’t tell you how much that helped me.”
  •  Less good: “I saw your new article in AJR. It must be nice to be friends with the editors!” [tip: resist all temptation to follow-up a compliment with an “it must be nice to…” or “I wish I had…”].

2. Personal

  • Good: “As an ethnographer, I rarely find quantitative research that taps into what I’m doing. But you really seem to ‘get’ the processes I’m seeing in the schools.”
  • Less Good: “Your work has decent face validity.”

3. Acknowledge Effort

  • Good: “Please tell me it took you all day to write that last paragraph – you completely nailed that civic reintegration idea.”
  • Less Good: “I’ve seen your blog. I wish I had so much extra time on my hands!”

4. Specific

  • Good: “I really liked your health disparities review piece, especially how you pulled in public health stuff – it was great for my prelim.”
  • Less Good: “I’ve read a lot of your articles” [As an old friend once said, “that’s how I know they’re lying – there aren’t that many of my articles to read!”]

5. Memorable

  • Good: “Smashing network diagrams!”
  • Less Good: “Nice slides.”

Don’t be surprised if the recipient of your compliment doesn’t know how to respond (usually, a simple “thanks” will do). We’ve been socialized to expect ulterior motives or to think our work isn’t worthy of kind words. But don’t worry about embarrassing those you compliment. As Erving Goffman pointed out, when a person blushes upon receiving a compliment, she may lose her reputation for poise but confirm a more important reputation for modesty.

Revere
Paul Revere by John Singleton Copley, 1768
Courtesy of MFA Boston & Wikipedia via Slate

An elegant design, compelling evidence, and a timely story rendered exceptionally well. Sociologist Kieran Healy’s wonderful post on using metadata to find Paul Revere (and/or Jack Black) is now attracting megareaders at Slate. The opening lines:

London, 1772. I have been asked by my superiors to give a brief demonstration of the surprising effectiveness of even the simplest techniques of the newfangled Social Networke Analysis in the pursuit of those who would seek to undermine the liberty enjoyed by His Majesty’s subjects. This is in connection with the discussion of the role of “metadata” in certain recent events and the assurances of various respectable parties that the government was merely “sifting through this so-called metadata” and that the “information acquired does not include the content of any communications.”

and,

I cannot show you the whole Person by Person matrix, because I would have to kill you. I jest, I jest! It is just because it is rather large. But here is a little snippet of it. At this point in the 18th century, a 254×254 matrix is what we call Bigge Data. I have an upcoming EDWARDx talk about it. You should come.

I won’t spoil the ending, but Dr. Healy’s explication is masterful, engaging important civil liberties questions while bestowing some serious geek cred to social network analysis. A good methods piece both intrigues and inspires, inviting the reader to pick up some new tools while reducing the real or imagined barriers to doing so. Why’d he write it? From today’s update:

I wanted to give non-specialists a sense of how the structural analysis of what’s being called “metadata” works, and to show in a fun but hopefully telling way how much you can get out of that approach. So I tried to emphasize that I was using one of the earliest, and (in retrospect) most basic methods we have, but one that still has the capacity to surprise people unfamiliar with SNA.