Replication and Extension Projects: Making Class More Interesting and Useful

This semester I taught the second course in my department’s quantitative methods sequence that is required for all of our graduate students: Advanced Data Analysis for the Social Science. Sociology departments around the country all have a pretty similar required sequence. In teaching the course this time, I tried to modernize it so that it would train students for the future (not just the present or the past).

One big aspect of this modernization was requiring students to complete a project where they replicate and extend an already published paper. Overall, this change was a big success, and I’d recommend that other classes also try it. In this post, I’ll share some of what worked about the project and how I will do it better next time. I’ve also made all of the materials that we’ve used available on the class website.

Why do a replication and extension project?

There are two main reasons that I decided to add a replication and extension project to my class: 1) it is great for the students and 2) it is great for the field. As students transition from consumers of quantitative research to creators of quantitative research, there are a variety of challenges that they encounter related to analyzing real, messy data. These challenges are hard to describe to students but are painfully familiar to anyone who has created a serious piece of quantitative research. I’ve come to believe that courses where students work exclusively with toy data—such as courses that I’ve taught previously—do not fully prepare students to become creators of researchers. In fact, this realization, that I was not preparing students for “real” research, was one of the most important pieces of feedback that I’ve received about my old courses.

In addition to helping my students, I hope that the replication and extension projects will help my field move to a system of open and reproducible research.

What exactly did the students do?

Just deciding the scope of the project was difficult. To me the most important things were that they should reproduce (or try to reproduce) a paper exactly; they should try to do something new; and they should not try to write a whole new paper. After my course, students take a yearlong course where write an entire empirical paper. I did not want to replicate that class. Rather, I wanted to give them focused practice on just one part of writing a paper: the data analysis.

I ended up with a six part structure.

Pick a paper
Get your plan approved
Reproduce the results exactly
Get feedback including from peers
Do something new
Get feedback including from peers

More information about each step is available on the class webpage.

How did it work?

As I said earlier, I think it was a big success. I was impressed by how much students seem to care more about their projects than they cared about their homework. I don’t yet have a way of getting detailed, real-time measures of student learning, but I would guess that they learned more per hour working on their projects than on their homework. Further, I would guess that the things that they learned on their projects were higher-order skills that are harder to acquire from a book or a video.

Also, the projects really enriched our class discussions. For example, when we were learning about multi-level modeling, the students who were working on a paper with multi-level modeling could explain that paper to their peers.

Ultimately the best measure of the success of the project will be the students experience doing their own research. Fortunately, they will do that next year, and I’ll do an evaluation at the end of that to see how it turned out.

What would you do differently next time?

Although it might all sound smooth here, there were a lot of rocky parts to this project. Here’s what I would do differently next time. I hope that these tips can save you and your students some pain.

More support choosing papers

By far the biggest problem was that some students picked papers that were too difficult given their background. The schedule calls for students to reproduce the results of a paper in six weeks, which is quite fast. Because of this, I required students to use projects where data is already public. But, still several students ran into problems. To address this, I’ve developed an improved format for the project proposals. This highly structure format might seem strict, but it would have alerted us to all of the papers that turned out to be too hard. I guess this is something that I’ll just get better at understanding over time.

More work pairing students

Students completed the project in pairs (similar to Gary King does it Gov 2001). This turned out to work well as many of the pairs were made up of students with complimentary skills. However, next time I would work a bit more to ensure that students with strong backgrounds in data analysis and coding were more evenly distributed across the groups. Pairs with two novice students really struggled at points.

More chances for the students to present their progress

Each team made a 5 minute presentation at the end of the semester about their project. Next time, I would offer students more opportunities to share their work. I think that this would motivate progress and promote collaboration across teams. I think that a better presentation schedule would be: halfway point (replication complete or nearly complete); three-quarters point (extension in progress); and end of the semester (everything complete).

Really push hard on replication by spring break

The schedule called for students to complete their replication by spring break, the midpoint of our semester. Honestly, very few of the groups really made this deadline, and many were far from it. This created problems later in the semester because it meant that many groups had little to no time for their extensions. Perhaps by choosing more reasonable papers and pushing hard, the students could have more time for an extension.

In designing this project, I benefited from the experience of a number of teachers who have assigned similar projects in their classes: Kosuke Imai, Nicole Janz, Gary King, Brandon Stewart, and Cristobal Young. I want to thank them for sharing their advice either in person or in writing. Finally, I especially want to thank our class TA Angela Dixon who helped our students with their projects and help me make this project more focused and more useful.

First published at Wheels on the Bus.

Active Learning Exercise: There’s Research on That!

There’s Research On That!” (#TROT) is a blogging project at The Society Pages that presents short summaries of classic and contemporary research relevant to current events as a resource for journalists and the public. While our archives are a great resource for students looking to kick-start a sociological research paper or for readers who want a fresh take on the news for their next cocktail party, the format of these posts also works great for class assignments! For this activity, students write their own TROT posts as an exercise in collecting, evaluating, and summarizing research.

Guidelines:

  1. Browse the “There’s Research On That!” blog to get a sense of how TROT posts are written. They usually start with a summary of a current event, then provide sociological ideas relevant to the matter at the hand. The author highlights one to four key ideas from the research and includes citations for each source with hyperlinks to the authors’ website and the publication information.
  2. Choose a recent news event or pop culture trend to analyze. Ask yourself, “What would sociologists have to say about this that the rest of the media may be missing?” Make a list of a few possible themes to investigate about the topic, such as the impact of social construction, institutions, or networks.
  3. Search for literature on your themes using an online database such as JSTOR, Sociological Abstracts, or Google Scholar. Select three to five pieces that speak to the themes or provide broader information about the topic.

***Start with major journals in the field—American Sociological Review, American Journal of Sociology, or Social Problems—but the search doesn’t end there! Books and publications focusing more narrowly on subfields, such as Journal for the Scientific Study of Religion, Gender & Society, or Deviant Behavior are always helpful. Also, remember to think broadly; sometimes a general article about the topic can be more helpful than one that fits the news story perfectly.

  1. Evaluate the research. Read the methodology sections of any journal articles, or find reviews of the books published in academic journals. Do you find the conclusion(s) convincing, based on the evidence provided? Is the methodology high quality? Which pieces provide the best takeaways for a general reader? Once you’ve considered the possible sources separately, consider how they will fit together in your post. From your list, pick the best two or three pieces to include in your article.
  2. Write a TROT post in the style of the website. Remember the main components: a short summary of the news, including links to media coverage so the reader can see what’s already been written; key takeaways you identified in the sociological literature; and citations of the research. Be prepared to discuss in class why you picked the sources you did.

TROT is an evolving project quickly becoming an integral part of TSP. Instructors are encouraged to contact the TSP staff (email stewa777@umn.edu) with the best TROT posts from this assignment for possible publication on TSP!

Intro to Sociological Methods Using Discoveries

Note: At the time this post was written, The Society Pages’ Discoveries were called The Reading List.

DISC clippedFindings about lifetime earnings, fertility, graduation rates, and gentrification are interesting all on their own, but how do sociologists go about studying these topics? To address this question for my Intro to Sociology class, I began with a 10-min. mini-lecture on “Methods” based on the info in Ch. 2 of Dalton Conley’s You May Ask Yourself textbook. (The students had SQ3Red this chapter for homework.) To kick it off, I reviewed how sociology was different from the other disciplines—our topic from the previous class, and then got into the new material about variables, samples, qualitative and quantitative, methods, etc.

So that the students could develop a fuller sense of the new concepts they learned, I created this methods sheet for the class to practice applying them. First, as an example, we answered the questions from the sheet about the first article in the Reading List Packet together as a class. Students were able to ask questions about the procedure and I could clarify the directions before they tried one on their own.

Then, students were instructed to get into small groups of 2 or 3. Each group was directed to work on answering the questions about a different Reading List article. When all groups had finished, each reported back to the class what the question, findings and methods were.

Why I like this activity:

  • it’s easy to differentiate if you have a heterogeneous class. The Reading List articles vary in difficulty, so you could assign easier articles to groups that struggle and more difficult ones to groups that need a challenge. Alternatively, you could differentiate by interest by letting each group claim which article to begin with.
  • if one group gets done way ahead of the others, they can move on to a different article, so nobody’s stuck waiting around for groups that need more time. If the whole class picks up the skills quickly, you could set a time limit and reward the group that finishes the most articles before time is up.
  • it’s social—at the beginning of the semester, students can get to know one another, and get comfortable working together and sharing answers with the class
  • having groups report back to the whole class provides a low-stakes way for the teacher to correct students’ misunderstandings in a way that benefits the whole class
  • students learn about the breadth of sociological inquiry while practicing their skills of identifying parts of a research article (and—perhaps more importantly—the parts of a research study)

 

Mediating Media Responses to Tragedy: Considering How Social Science Could Influence Policy

The NonViolence Project takes its symbol from a sculpture inspired by the shooting death of John Lennon. Photo via nonviolence.com.
The NonViolence Project takes its symbol from a sculpture inspired by the shooting death of John Lennon. Photo via nonviolence.com.

Horrible events, such as mass shootings, typically gain a lot of media attention, with fear and political outrage not far behind. Social scientific knowledge about topics like violence, gun control, and mental illness, however, is often obscured or excluded from these reports and calls for action. This activity, which can be done as a group or individually, helps readers think about how social scientific evidence can influence policy:

  1. Browse the Internet to gather two or three news stories from the weeks following a recent mass shooting in the United States.
  2. What claims are made in these stories about the causes of mass shootings?
  3. What calls for change are made by victims’ families, politicians, experts, or others?
  4. What policies are suggested to address mass shootings?

Next, read “A Broader-Based Response to Shootings” by Chis Uggen and think about how social science evidence compares to media reports. What does the evidence suggest we should be doing to address these crimes?

Politics and Power

A still from the Public Enemy video for "Fight the Power." Chuck D, holding the bullhorn, has since become an outspoken agitator and public figure working toward equality and political participation.
A still from the Public Enemy video for “Fight the Power.” Chuck D, holding the bullhorn, has since become an outspoken agitator and public figure working toward equality and political participation.

In our volume The Social Side of Politics, Vincent J. Roscigno’s article, “Power, Sociologically Speaking,” serves as the lead essay. In it, we learn that you can’t discuss politics without discussing power. This activity helps highlight the point for students and instructors alike.

Guidelines for the facilitator:

  1. Make four signs labeled “agree,” “strongly agree,” “disagree,” and “strongly disagree.” Hang one sign in each corner of the room.
  2. Tell participants that you will read a statement about power (listed below). After you read the first, participants should move to stand under the sign that most closely reflects their reaction to the statement.
  3. After participants have assembled under the signs, ask each group to discuss why they picked that position and choose a spokesperson to explain their rationale to the entire group.
  4. After each group presents its opinion, as participants to return to the center of the room, then disperse, again, to the sign that most closely represents their reaction to the same statement. If they choose, participants can change their position. Ask a few to explain whether and why they changed—or did not change—their position after hearing out the other groups.
  5. Repeat for each statement, adding or subtracting to alter the length of the exercise.

Statements about Power:

  • Power corrupts.
  • You can’t get anything done without power.
  • Power is connected to race (class/gender/etc.).
  • People or organizations that want to change things in their community should seek power.

Desistance and Reentry: An activity for the LCD classroom

A scene from "The Road from Crime"
A scene from “The Road from Crime”

In our volume Crime and the Punished, we featured our interview with sociologist-filmmakers Shadd Maruna and Fergus McNeill. This activity builds from their film:

How and why people stop committing crime is an important question. “Discovering Desistance,” by Sarah Lageson and Sarah Shannon describes how two social scientists “co-created” “The Road from Crime,” a film about desistance from the perspective of former prisoners and the practitioners who work with them. Watch the 50-minute film as you consider these discussion questions:

  1. In what ways might the criminal justice system promote reoffending?
  2. According to the filmmakers, desistance is both an “internal” and an “external” process. Where do you see internal and external processes in the film.
  3. What punishment policies might be changed, added, or abandoned to better promote desistance?
  4. Most people who work in the criminal justice system have never been convicted of serious crimes. How might the system be different if it incorporated more input from people who had been punished under it?

After watching the film, imagine that you are a social worker in a community to which many ex-prisoners return. What resources do you think you’d need to address community needs and help former inmates desist from crime? What community leaders or organizations would you need to enlist for support? Discuss your thoughts in a group and draw up a list of the “stakeholders” whose voices are critical for designing your policy.

How to Use #Selfies as Sociological Exercises

Aoki-chainsmokers_v3Linda Catalano is a sociologist at Queens College and Hunter College. Follow her on Twitter at @SocThing.

Selfies, I’ve found, are a terrific way to begin to get across George Herbert Mead’s distinction between the “I” and the “me,” which students can find difficult to grasp and tend to resist. I’ve developed an exercise that incorporates selfies which works fairly well, but I suspect that there are even better ways of using selfies that draw out more of Mead than I’ve been able to do.

I’ve been using a textbook (Edles & Applerouth 2010) with several selections from Mind, Self & Society (Mead 1934) in which Mead outlines his famous concept of the self as incorporating two phases, the “I” and the “me.” I was surprised to discover that my students have considerable trouble with this notion, and given the choice, avoid questions about it on exams.

Why students resist. Students don’t like Mead for many reasons. Aside from the fact that they often have trouble with his long and convoluted prose, students tend to have several conceptual difficulties. First, they have trouble with the way that, for Mead, social reality is all process. Students tend to feel their conceptual ground turn into quicksand, that they have nothing to hang on to. more...

Bringing the “Beautiful Game” to the Classroom

Photo by Steve Tulk via flickr CC
Photo by Steve Tulk via flickr CC

The following is a guest post written by Kyle Green and Alex Manning. Kyle and Alex are sociology Ph.D students at the University of Minnesota. Kyle is a member of The Society Pages graduate board and co-host of the Office Hours podcast. His research focuses on culture, sport, gender, and the body. Alex researches race, youth, parenting, and sport.

This World Cup, soccer is on American’s minds at levels never reached before. Fans are filling bars, coffee shops, and even massive stadiums to cheer on the US national team. World Cup fervor has led many in the U.S. media to ask the once every four-year question, “has soccer made it in America?” Large television numbers, a sizeable number of American fans supporting in Brazil (Americans bought 7% percent of world cup tickets, only trailing Brazilians), large participation numbers, and increased youth consumption of the game, have all contributed to public discussion about the game’s popularity and place in the United States.  While there has been much excitement surrounding soccer and the World Cup, some have reacted to the popularity of the tournament with fear, dismissal, and outrage (here and here).

Audience: This activity would work well in a number of courses including Introduction to Sociology, Race, Class, & Gender, Sport & Society, and Sociology of Consumption.

Summary: In this activity the class works together drawing on personal experiences and associations with soccer to think about the social spaces the sport fills and to connect the participation and consumption patterns of the sport to larger social trends. In doing so the students will use their sociological imagination to begin to understand the fervor, both positive and negative, surrounding the World Cup. more...

The Danger of a Single Story

This year I taught Introduction to Sociology. In order to discuss the power of discourse in society, I showed by students Chimamanda Adichie’s 2009 TED Talk called “The Danger of a Single Story“. My students were enamored. We had a fascinating and engaging discussion about single stories and the ways in which they affected my students’ lives and their engagement with the world around them. As a result of this phenomenal class, I developed the following assignment that I thought other sociologists would like to adapt to fit their courses.

Assignment Description and Instructions:

Chimamanda Adichie passionately and clearly teaches us the “danger of a single story” in her 2009 TED Talk. (You can find it here: http://www.npr.org/2013/09/20/186303292/what-are-the-dangers-of-a-single-story). Adichie demonstrates the ways in which our society is a collection of social stories or narratives, the most pervasive and controlling of which are/were manufactured by people with social power (the power elite).

Single stories can include stereotypes, ideologies and, what sociologists call, cultural hegemony. Stereotypes are overly simplistic generalizations about a subgroup of peoples. Those that “stick” often are constructed by people with power and used to limit opportunities for the stereotypes’ subjects. Ideologies are sets of ideas that shape how people make sense of the world around them. Depending on the social power of those holding and employing these ideologies, they can have significant impact on social structures and the life chances of others. Cultural hegemony is a system beliefs, norms, and values, shaped by the ruling-class, that justifies the status-quo as natural or normal, and thus makes it invisible. These discourses shape what is knowable and sayable in any given context.

For your papers, you will select a societal single story and analyze it. The first paper will examine a stereotype, the second an ideology, and the third a hegemonic narrative. For each, you will explore the story, its origins, its functions, and its impact on society. You will then examine the alternative stories: those told by the victims of the single story and/or those who are able to see through the discursive fog. Finally, you will propose ways to change the story both in your daily life and on a broader scale. As you move through these projects, also reflect on the ways in which stereotypes, ideologies, and hegemonic narratives are intertwined/not clearly separated. more...

Summer Lovin’ and the Sexual Double Standard

Grease
“Hooking up” on college campuses has been the focus of a number of debates, both in the media and in sociology, over the past several years. Some argue that casual sexual encounters are detrimental to women’s self-esteem but that hookup culture “hurts boys too.” Others assert that hooking up, which has supposedly replaced dating on college campuses, is leaving both men and women “unhappy, sexually unfulfilled, and confused about intimacy.” Many sociologists, including Paula England and Lisa Wade, have been in the midst of these cultural debates about hooking up and its effects on young adults.

As hookup culture on college campuses seems relevant (or at least an interesting topic of discussion) to students in my classes, I spend a class during the weeks on gender and sexuality addressing the sociological debates about hooking up and casual sex. There is no shortage of readings that could be assigned on this topic, including many popular media articles. One of the readings that I always assign, “Is hooking up bad for young women?” from Contexts a few years ago by Elizabeth Armstrong, Laura Hamilton, and Paula England, does a great job of outlining the debates about hookup culture that continue to be relevant. (This article was also the focus of another Teaching TSP post several years ago). I also have them read Hanna Rosin’s article from The Atlantic and Lisa Wade’s response from Soc Images.

One of the points of discussion, the sexual double standard, is repeatedly brought up with frustration by female students in my classes. Why are women who have casual sex considered sluts while men are practically given a medal for hooking up? As I initially struggled to navigate these discussions, I turned to an unlikely place for guidance, the classic musical Grease. Who hasn’t belted out “Summer Lovin’” on the dance floor with friends?

As Sandra Dee (Olivia Newton John ) and Danny Zuko (John Travolta) trade off singing about their summer romance, they seem to be describing two different relationships. Sandra Dee tells her friends, “He was sweet, just turned eighteen,” as Danny dishes, “She was good, you know what I mean?” This song, it turns out, is a perfect illustration of different gender norms when it comes to sexuality.

For the class activity, I have students divide a sheet of paper into two columns with one side for Sandra Dee and the other for Danny Zuko. As they watch the Summer Lovin’ footage from Grease, I ask them to take notes in each column of quotes or themes from the song that reflect gender norms and the sexual double standard.  Afterwards, we make a list on the board of the stark differences in gender norms around sexuality.

I use this activity as a jumping off point from which to discuss the roots of the sexual double standard and some of the issues with hookup culture. Some questions that we consider in class discussion include:

1)     How could hooking up be considered bad for men or women or both? What could you argue are the benefits to hooking up?

2)     How do gender norms operate within hookup culture (Lisa Wade’s article does a particularly good job of outlining this issue)?

3)     Does hookup culture have the potential to disrupt the sexual double standard or to change gender norms?