Scholarly publications are not necessarily free from error. Researchers like Mark Regnerus have operational problems that skew their claims. Others publish with typos. And still others make mistakes in translating data to graphs, tables, or other infographics. Peer review can only catch so much, because reviewers don’t often have access to the full data set, at least not when dealing with qualitative data. Or, in the case of Regnerus’ Social Science Research publication, blinded reviewers overlook egregious errors in conceptualization and conflicts of interest in project funding (see here for a nuanced critique).

About a year ago, we both discovered an error in a 1976 research note published in the American Journal of Sociology that resulted in an Erratum in the journal’s May 2018 issue. The error appears in a really interesting article by Sociologist Dwight E. Robinson on shifts in men’s facial hair fashions over the course of 130 years in London. Robinson tracked representations of facial hair as a case study of fashion trends as measurable bits of culture. Comparing shifts in men’s facial hair to shifts in women’s skirt lengths, for example, he made claims that “men are just as subject to fashion’s influence as women” (here: 1133).

In the research note, Robinson calculated the relative frequencies of five different styles of men’s facial hair (clean shaven, moustaches, sideburns, moustache & sideburns, and full beards), and different combinations of these styles, from images published in the Illustrated London News between 1842 and 1972. This project shows dramatic shifts in configurations of men’s facial hair over the period studied, with a spike in different styles at different times but an overall decline in facial hair since the late 1800s. Robinson also reported on this shift in Harvard Business Review a year prior, in an article comparing this trend to still more cultural shifts in fashions.

Plotting his findings allowed Robinson to visualize this shift over time, and visualizations help to more readily appreciate the cyclical nature of cultural shifts in fashion (like changes in the popularity of baby names, for instance). They help make discernible something that might be otherwise difficult to appreciate. Below, we’ve stacked all the relative frequencies in a chart to display this shift (also in Sociology NOW, 3e, Chapter 4). It’s really an incredible change, and such a neat way to talk about shifts in fashion. Some fashions have short cycles (like styles of clothing, for instance), while fashions associated with other things (like popular baby names) have longer cycles. Facial hair fashions, according to Robinson’s research, appear to follow a fashion cycle more similar to baby names than to styles of clothing.

But… in the American Journal of Sociology article, there are a collection of errors in the Appendix table from which we collected these relative frequencies. These errors are reproduced in both the  AJS and Harvard Business Review. Robinson may not have realized these mistakes because he plotted shifts in facial hair styles on separate graphs both publications (see images below).

The graphs are produced from relative frequencies of a raw count of men’s facial hair styles in each year of published issues of the Illustrated London News. When we requested Robinson’s submission files from the American Journal of Sociology to consult when assessing the error, they no longer had them. This would have been in hard copy and that filling system, we were told, did not include his submission materials. We also tried to collect submission files from Harvard Business Review, which no longer has the files. Because of this, the Editorial Board at AJS decided they were unable to correct the errors in an erratum; they did agree to at least publish something stating that errors were indeed made. After all that investigation, we ended up with this Erratum:

This erratum is a bit non-committal. But it was what the journal was willing to print. Don’t get us wrong, these errors don’t have the same policy implications as the egregious Regnerus study that suggests children of gay parents don’t meet markers of success similar to kids’ of straight parents. We do feel, however, that the errors can and should be corrected with the available information.

Robinson’s errors appear to most likely be the result of mistakes make in calculating something simple: relative frequency. Because Robinson included all of the figures in the appendix, he allowed us to calculate these frequencies ourselves for verification. Journals should do this when they can, to make scholarly claims more transparent and to offer other scholars data that could be used in different ways, to perhaps answer different questions. Indeed, more journals are including data files as a part of the available materials for download, now that things are online. Below is the Appendix from the article published in the American Journal of Sociology.

The errors in the table (reproduced in the figures in both publications) are associated with the years: 1844, 1860, 1904, 1916, and 1959. In each case, the relative frequencies are miscalculated in the table.

• 1844: The relative frequency of clean shaven should be 30%, not 47%.
• 1860: The relative frequency of beards should be 40%, not 39%.
• 1904: The relative frequencies of moustaches and beards should both be 34%, not 37% and 32% (respectively).
• 1916: The relative frequencies of clean shaven and moustaches should be 34% and 65%, not 33% and 64% (respectively).
• 1959: The relative frequencies of clean shaven and moustaches should be 78% and 22%, not 74% and 21% (respectively).

These errors do affect what the graphs look like. If they were corrected, we would see a slight rise in the popularity of representations of men with mustaches in the late 1950s. Now, is that a significant difference? Not really. Clearly, we went to more trouble here than necessary. But identifying (and correcting) research errors is as important to maintaining scholarly integrity as is conducting meticulous reviews of research before it’s published. Accountability is key to making sure we, as scholars, continue to understand research as a communal process that takes seriously the integrity of research, from the smallest details to the biggest biases.