{"id":489,"date":"2016-08-03T20:54:48","date_gmt":"2016-08-03T20:54:48","guid":{"rendered":"https:\/\/thesocietypages.org\/edsociety\/?p=489"},"modified":"2016-08-03T20:54:48","modified_gmt":"2016-08-03T20:54:48","slug":"one-thing-i-learned-by-editing-sociology-of-education","status":"publish","type":"post","link":"https:\/\/thesocietypages.org\/edsociety\/2016\/08\/03\/one-thing-i-learned-by-editing-sociology-of-education\/","title":{"rendered":"One Thing I Learned By Editing Sociology of Education"},"content":{"rendered":"

\"quality<\/a>I very recently finished my service as editor of Sociology of Education<\/em>(SOE)\u00b9\u2014the global flagship journal of this subfield, one of the best journals in both sociology and education research. In that capacity, I read nearly 800 of your new and revised manuscripts\u2014roughly one every weekday for three years\u2014but had room to accept for publication only about 50 them.<\/p>\n

Eric Grodsky asked me to share with you my sense of the subfield after reading all of those manuscripts. I am going to give you my honest assessment because I care deeply about education research.\u00a0 I believe that as a subfield we are capable of making meaningful contributions to the scientific understanding of what education is, how it works, for whom it works and for whom it does not work, and what our society can do to improve it.<\/p>\n

In short, I do not believe we are living up to our collective potential.<\/p>\n

When I started as editor, I worried that the biggest challenge would be this: How will I choose from among all the great submissions?<\/em> How will I select just the four or five best articles to publish in each quarterly issue?\u00a0 That worry was based on my sense that a high percentage of submissions would be\u2014or could, with modest revision, become\u2014publishable.<\/p>\n

In fact, I consistently had exactly the opposite<\/em> problem. For each of the 12 issues I published, I struggled to find four high-quality publishable manuscripts. During my first year as editor, SOE had an extra page allocation from ASA\u00b2; I did not fill all the extra pages because there were not enough acceptable papers. In all three years, I pressed authors to write more concisely so that I could fit a fifth article in each issue; only once did I succeed in finding five publishable papers\u00b3. I am very<\/em> proud of the articles that appeared in SOE during my three years as editor\u2014and of their topical, methodological, and theoretical diversity\u2014but I leave the journal deeply dismayed at the state of the subfield. I am somewhat dismayed that there were so few high-quality papers.\u00a0 I am more dismayed that there were so many lesser-quality ones.<\/p>\n

Most of the papers that I read had one or both of two basic problems:<\/p>\n

First, a large percentage of papers had fundamental research design flaws. Basic methodological problems\u2014of the sort that ought to earn a graduate student a B- in their first-year research methods course\u2014were fairly common4. (More surprising to me, by the way, was how frequently reviewers seemed not to notice such problems.) I\u2019m not talking here about trivial errors or minor weaknesses in research designs; no research is perfect. I\u2019m talking about problems that undermined the author\u2019s basic conclusions. Some of these problems were fixable, but many were not.<\/p>\n

Second, and more surprising to me: Most papers simply lacked a soul\u2014a compelling and well-articulated reason to exist. The world (including the world of education) faces an extraordinary number of problems, challenges, dilemmas, and even mysteries.\u00a0 Yet most papers failed to make a good case for why they were necessary. Many analyses were not well motivated or informed by existing theory, evidence, or debates. Many authors took for granted that readers would see the importance of their chosen topic, and failed to connect their work to related issues, ideas, or discussions. Over and over again, I kept asking myself (and reviewers also often asked): So what?<\/p>\n

I gradually came to understand that (1) many authors just hadn\u2019t yet fully thought through the \u201cso what?\u201d questions and (2) many authors were submitting papers long before they had fully worked through crucial issues related to research design, quality of evidence, and coherence of argument. They didn\u2019t do a great job of motivating their questions because they weren\u2019t yet fully sure how their work fit in the larger scheme of things.\u00a0 They hadn\u2019t thought through the \u201cso what?\u201d of their findings because they hadn\u2019t had time to fully make sense of them.\u00a0 They made assumptions or mistakes in their research design and analyses\u2014just like everyone<\/em> does in the early iterations of a project and paper\u2014but they submitted their papers anyway.<\/p>\n

I think there are two related explanations for why the vast majority of SOE submissions over the past three years were simply not all that great and did not live up to their potential.\u00a0 First, junior scholars are under far too much pressure to quickly publish lots and lots of papers. The problem is that it takes time and thought and trial and error to generate a really compelling research question and to deeply understand how to articulate the importance of that question. Then, it takes more time and thought and trial and error to come to a really convincing answer to that question, to fully develop the argument or story, and to write a manuscript that does justice to the value of the research. Unfortunately, junior scholars do not have enough time or enough room to think or make missteps. Graduate students must publish frequently\u2014while still in graduate school\u2014to complete for fellowships and professional jobs. Assistant professors must publish early and often to get tenure; associate professors must do the same to \u201cmake full.\u201d\u00a0 In each case, the pressure is to work quickly and efficiently. They have to get papers out for review as soon as possible, even if they really haven\u2019t had the time to think them through or sharpen their argument or evidence. Quantity matters most, not quality. The incentive is to publish a dozen \u201cjust OK\u201d papers rather than to publish four or five well-developed and higher-quality papers.\u00a0 The collective result for the subfield: a mass of underdeveloped papers that often have some potential but really aren\u2019t acceptable for publication in high-quality journals5.\u00a0 These are papers that exist for the sake of being papers because the author is under pressure to write lots of papers.\u00a0 I sincerely empathize with those authors\u2014I once was one, and I socialize and work closely with many of them. Senior scholars in the subfield bear the responsibility for this state of affairs; they are in a position to change it.There simply has to be a better way to run a profession<\/em>.<\/p>\n

Second, junior scholars are not as well trained, mentored, or supported as they need to be. Partly, this goes back to the pressures to publish early and often. But partly, this is the collective fault of graduate training programs and junior faculty mentoring programs (and thus, again, of the senior scholars who lead those programs). Collectively, we simply need to do a better job of training, mentoring, and supporting junior scholars. Standards and expectations need to be higher, but junior scholars need to be given the resources\u2014including time and mentoring\u2014required to meet them. The subfield needs to be more innovative and energetic in designing creative ways to improve training and scholarship.<\/p>\n

The modal SOE submission during my three years as editor was written by a junior scholar, had non-trivial methodological problems, and lacked a compelling theoretical or substantive rationale for having been written. This is despite the fact that the authors were well-intentioned, were extremely bright, and worked hard at their craft. Virtually every paper had some<\/em> kernel of real potential in one way or another\u2014some creative idea, some new insight, some thoughtful new approach\u2014but far too often that potential was not realized. All of this represents, to my mind, a waste of resources\u2014including authors\u2019 time and energy and frequently tax-payers\u2019 money. And, it doesn\u2019t help any of us move closer to addressing the problems, challenges, dilemmas, and mysteries that we all face in education and our broader society.<\/p>\n

Nobody asked for my views on how to fix these problems, but here they are anyway. First, sociology departments, education schools and their kin need to de-emphasize thequantity<\/em> of published papers in making hiring and promotion decisions. Second, those units and fields need to primarily reward the quality<\/em> of publications. Third, we all need to be more energetic, more creative, and more collective-minded in training, mentoring, and supporting junior scholars.\u00a0 To be clear: The onus of implementing these badly-needed changes falls on senior<\/em> scholars.\u00a0 Junior scholars are appropriately and understandably focused on their own careers.\u00a0 It falls on senior scholars to transform incentive structures\u2014rewarding quality over quantity\u2014and to provide junior scholars with the time and support they need to do better research and write better papers.<\/p>\n

ENDNOTES<\/strong><\/p>\n

1 Although my name is on the masthead as editor for the rest of 2016, incoming editor Linda Renzulli took over the day-to-day operations of the journal on July 1.
\n2 That\u2019s right: In the 21st century ASA still thinks in terms of printed page allocations and mailing costs.
\n3 One possibility is that I was too harsh a judge of submissions.\u00a0 Maybe, but: About one in seven authors of new submissions was invited to revise and resubmit their manuscripts, most revised papers were subsequently accepted for publication, and I did ultimately publish at least four papers per issue.
\n4 The most common questions I wished authors had asked themselves before they submitted their papers included: Is this the best group of people to study in order to achieve my research objectives?\u00a0 How should I measure key concepts, and what are the relative strengths and weaknesses of those measures?\u00a0 Does the evidence I have presented speak directly to my questions or hypotheses?\u00a0 Does that evidence adequately support my arguments, claims, and conclusions?\u00a0 Am I making causal claims, and if I so does my evidence support such claims? If I am comparing multiple groups, have I actually compared those groups?
\n5 This is not to say that as editor I expected papers to be acceptable and publishable right out of the starting gate.\u00a0 I invited the authors of about 13% of new manuscripts to revise and resubmit their papers based on reviewers\u2019 and my own comments.\u00a0 I know that a big part of the job of a journal editor is to identify promising papers and help them become higher-quality published articles.\u00a0 Like I said, I\u2019m very proud of the papers I published.<\/p>\n

[This article is reposted with permission from Rob Warren’s blog<\/a>.]<\/p>\n","protected":false},"excerpt":{"rendered":"

I very recently finished my service as editor of Sociology of Education(SOE)\u00b9\u2014the global flagship journal of this subfield, one of the best journals in both sociology and education research. In that capacity, I read nearly 800 of your new and revised manuscripts\u2014roughly one every weekday for three years\u2014but had room to accept for publication only […]<\/p>\n","protected":false},"author":1952,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[],"class_list":["post-489","post","type-post","status-publish","format-standard","hentry","category-uncategorized"],"jetpack_featured_media_url":"","_links":{"self":[{"href":"https:\/\/thesocietypages.org\/edsociety\/wp-json\/wp\/v2\/posts\/489","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/thesocietypages.org\/edsociety\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/thesocietypages.org\/edsociety\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/thesocietypages.org\/edsociety\/wp-json\/wp\/v2\/users\/1952"}],"replies":[{"embeddable":true,"href":"https:\/\/thesocietypages.org\/edsociety\/wp-json\/wp\/v2\/comments?post=489"}],"version-history":[{"count":3,"href":"https:\/\/thesocietypages.org\/edsociety\/wp-json\/wp\/v2\/posts\/489\/revisions"}],"predecessor-version":[{"id":493,"href":"https:\/\/thesocietypages.org\/edsociety\/wp-json\/wp\/v2\/posts\/489\/revisions\/493"}],"wp:attachment":[{"href":"https:\/\/thesocietypages.org\/edsociety\/wp-json\/wp\/v2\/media?parent=489"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/thesocietypages.org\/edsociety\/wp-json\/wp\/v2\/categories?post=489"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/thesocietypages.org\/edsociety\/wp-json\/wp\/v2\/tags?post=489"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}