GUEST POST by Abigail A. Sewell

Recently, The New York Times published a write-up of a study documenting a surprising “non-finding”: racial bias in use of lethal force by police is non-existent.

For those this finding surprised, the response has been swift, vociferous, and cross-disciplinary.

Who’s right?

My bet is on the community, given my expertise on the health effects of police surveillance published in interdisciplinary journals like Journal of Urban Health and Social Science and Medicine.  You can also look at the overwhelming research on race and policing showing stark racial differences not only in treatment and interactions, but also in police killings.

Research compiled by The Race and Policing Project indicate that such police behavior affects a host of outcomes beyond health, including attitudes minorities hold towards the criminal justice system. The invisibility of this research is why a group of scholars, coordinated by myself, have come together to make available such research to the public in ready-access format.

Fryer’s discovery of a null relationship comes from a rather classic research design of racial bias using precedents set by economists. In fact, it is a pretty beautiful study in its meticulousness in setting the parameters for identifying racial inequality in police use of lethal force that is not due to statistical discrimination. Further, it is impressive to me because of its reliance on team science, where a researcher employs the assistance of a large group of people to enact the research design and analysis for a single research question. Lastly, it partially confirms prior findings on racial bias in police use of nonlethal force, providing the appearance that its “finding” of null relationships are in fact reliable and valid.

But that’s where my compliments end. Interpreting Fryer’s “non-finding”  (i.e., there is no racial differences in police killings) as valid is problematic because the study suffers from many flaws due to its research design – the more serious of the two being choice of denominator (which was, people killed with arrest warrants for shootings deemed justifiable by the researcher) and “over-control” (he adjusts baseline estimates for variation in over 200 social factors in his study). Economists have a notorious habit of trying to “explain away” a pattern, but statisticians know that the more controls one puts in a regression model, the less degrees of freedom one has to represent reality.

At the very least, even though baseline estimates (average group differences) may show no evidence of racial inequality, Fryer could have used propensity score matching to account for the fact that racially marginalized populations are more likely to be stopped in the first place than their white counterparts. Even thinking about moderated models may have helped the interpretation of baseline non-findings, meaning that there is heterogeneity in the nature of racial differences across subgroups of the population. More sophisticated methods would have used multilevel models to account for variation in the probability of being stopped across place and race, such as I do in my research.

Still, the real, more troubling problem with the Fryer study is that it is steeped in academic disciplinarianism. Academic disciplinarianism refers to adhering to the principles and practices of being an expert in a specific field of study. In doing so, you may adopt the data, measures, and methods of the discipline you affiliate with at the sacrifice of methodological standards that may be more effective in answering your intended research question. Moreover, you rely on the intellectual efforts of people in your discipline, who may disproportionally come from similar backgrounds and thus carry a shared understanding of how research should be conducted and, ultimately, suffer from groupthink. Last, you write in the language of your chosen discipline, assuming that people understand what you are talking about when you say words like “racial bias” and “statistical discrimination” and discarding critiques from people of other disciplines and perspectives of how to conduct research.

Mind you, interdisciplinary research is difficult for all the reasons we know –communication barriers, conflicts in methodological approach, and competing definitions, to name a few. However, research on race and policing is prone to interdisciplinary efforts and, even protected, by such an approach because academic treatments of both “race” and “policing” vary widely based on the discipline in which one is trained. That alone does not necessitate Fryer to do interdisciplinary research; however, for three reasons, it places limitations on interpreting null findings.

First, because Fryer’s study relied on methods familiar to and accepted by economists, it failed to adopt a research design best fit for small-N research – that is, QCA, or qualitative comparison analysis. QCA enhances detection of variation in instances whether standard errors are unreliable due to sample sizes. Fryer’s study of racial bias in police use of lethal force relied on some 300 cases in one city. Ideally, more cases would have been collected, and more cities would have been examined.

However, let’s assume that the data from Houston is indeed the best data available: Econometric techniques produce estimates of difference that are more unreliable in situations where sample sizes are small. There is a positive correlation between reliability and validity, such that unreliable estimates have a greater chance of being untrue in actuality. This is akin to a Type II error, where random sampling does not allow for you to find evidence of patterns in the data that actually exist in the real world. Moreover, it is interesting that he chose a single city, when studies show that racial disparities in police encounters are greatly reduced by considering variation in police encounters across place, according to Fryer’s own estimates.

Second, because Fryer relied on the intellectual efforts of other economists, he pigeon-holed himself into the pitfalls of the economics discipline. This is not an indictment of an entire discipline. Rather, this is an advocacy of interdisciplinary research teams. Now, the exact discipline of each of his research team members was not released, but I am going to guess by the near racial monotony of the research team posted on The New York Times that they’re mostly, if not all, economists. Economics as a discipline has both under-recruited and under-valued the economic thinking of black folks. Many of the top economics departments in the nation are all-white, and black economists are under-cited in academic publications. Research that documents racial bias in economic outcomes tends to be rejected by top-journals, if accepted by a journal at all. These are facts, not opinions.

Third, a conceptual focus on racial bias by economists interested in understanding the roots of racial inequality requires many judgment calls to be made by the researcher, including most importantly what types of lethal police behaviors are justifiable. There is something akin to groupthink going on among economists in their narrow focus on racial bias, rather than racial disparities. Fryer’s goal in his study of lethal use of force by police was to rule out the possibility of statistical discrimination, which requires a different view of the data than simply examining racial disparities in lethal use of force. It requires for the researcher to “rule out” competing explanations and identify reasonable conditions under which discrimination is not a reflection of bias.

As a race and racism scholar steeped in the structural tradition, I am not of the mindset that discrimination occurs without bias. The bias may be reasonable or implicit, but it still exists – and that is a problem. And, that problem is structural in nature, rooted in anti-blackness ideology that pervades the entire system of social relations in which we exist.

Moreover, I think we live in a dangerous world if we condone police violence as societally acceptable, and I think many scholars in and outside of economics would agree with this. However, when all you do is converse with other people who think just like you, then you get very good at what you do. It is particularly dangerous to assume that everyone who is stopped by the police is actually guilty of a crime, as is done in the Fryer study. For instance, Philando Castile was stopped over 50 times for suspicion of offenses and never charged for anything, indicating that he was never guilty of anything. The suspicion, however, was created because of his blackness, because of police and societal expectations of crime as “black”, because of anti-blackness ideologies rooted in policing institutions ties to slave patrols.

Simultaneously, in this case of academic disciplinarianism, one also becomes less effective at speaking to the reality of the situation: Black and Latino men face a higher exposure to police surveillance, and all of its ramifications, than do White men. We can splice the data however we want to, but just because economic methods for “discovering” and “verifying” racial bias are employed, does not mean that racial disparities are not present.

Rather than conducting research in academic disciplinary silos, we who are interested in work at the intersections of race and policing need to work across the disciplinary divides. Yes, interdisciplinary work is hard. However, technology has made it increasingly easier to work in teams with people, like I am doing with my research on race, policing, and health, which is a product of a collaborative effort between public health scholars and sociologists. In addition to using technology to bridge the challenges of communication, we should also deposit our work in interdisciplinary repository, such as SocArXiv, to increase the reach of our work across the disciplinary aisles.

Abigail A. Sewell, Ph.D.

Assistant Professor, Department of Sociology, Emory University

Vice Provost’s Postdoctoral Fellow, Population Studies Center, University of Pennsylvania

. . .

Teach well, it matters.