In a recent Boston Globe story, diversity consultants and social scientists debated the effects of workplace diversity training.  While diversity training programs are a common job requirement these days, they may look very different from one company to the next:

The courses vary widely, in content and duration and method and philosophy: Some are short videos followed by structured discussions, some are multiday retreats, some are informational, teaching participants about their “diversity circle” and the difference between a generalization and a stereotype, others focus on role-playing. But they all promise to help people better navigate the fault lines of race, gender, culture, class, and sexual orientation that can divide co-workers and unsettle offices.

Opinions about the programs are also varied, and good social science evidence for either side of the debate has been scarce:

Such programs have always been controversial, with critics arguing that they’re unnecessary and needlessly politicize the workplace. But despite the growth and prevalence of diversity training, there have been few attempts to systematically study it.

Now a few social scientists are taking a hard look at these programs, and, so far, what they’re finding is that there’s little evidence that diversity training works.

Research by a team of sociologists on more than 800 companies over three decades has found that the best diversity training programs make little difference in who gets hired and promoted, and many programs actually decrease the number of women and minorities in management.

“Even with best practices, you’re not going to get much of an effect,” says Frank Dobbin, a Harvard University sociology professor on the research team. “It doesn’t change what happens at work.”

Diversity consultants are confident in their programs, claiming social science research in this area can’t accurately measure the impact of the training they deliver, generalizes unfairly, and rarely offers solutions to the problems it identifies:

Practitioners and some scholars disagree, arguing that, while there have been some unsubstantiated claims and overhyped “innovations” in diversity training, the field as a whole has begun to figure out what works. The changes that training triggers can often be subtle, defenders argue, and, in a setting as dynamic and stubbornly multivariate as the workplace, it’s all but impossible to come up with the clear, falsifiable evidence social science demands. The poor results that do show up in broad-based studies, they say, are due to companies whose commitment to diversity training programs is merely pro forma, and who see training as just a way to protect themselves from lawsuits.

“My experience is that a lot of these studies make good points, but they tend to fall into one particular trap,” says Howard Ross, a leading diversity consultant. “When we talk about diversity training as a megalith, it’s similar to saying, ‘Are restaurants good places to eat?’ The answer is ‘yes’ and ‘no,’ depending on the restaurant.”

Critics, on the other hand, argue that today’s practitioners are unlikely to be converging on a set of best practices, since the field is characterized by divergent, even contradictory approaches to the same set of problems. To critics, the proponents are simply mistaking the fact that people feel better about themselves after training for real results. Just because people think they’re less prejudiced doesn’t mean they are. Indeed, with something as subtle and reflexive as bias, we’re often our own worst judges

Dobbin and his colleagues have designed their research to address the potential alternatives to conventional diversity training programs practitioners often call for:

“We were increasingly frustrated by the fact that we know a lot about what kinds of disparities there are in organizations, and what kind of disadvantages women and minorities faced, but we know almost nothing about how to how to reduce them,” says Alexandra Kalev, a sociologist at the University of Arizona.

Several years ago Kalev, along with Dobbin and Erin Kelly of the University of Minnesota, set out to see what works. As a measure of program success, they looked at the number of women and minorities in a company’s managerial ranks – a much more concrete metric than the surveys of employee attitudes that many other studies relied on. The researchers drew on 31 years of Equal Employment Opportunity Commission data, specifically the annual reports that companies file detailing their racial and gender makeup. The sociologists then surveyed 829 of those companies on what diversity programs they had and when they instituted them. The results were described in a 2006 study, and in another paper that Kalev and Dobbin are currently writing.

The researchers found that while diversity training was by far the most popular approach, it was also the least effective at getting companies to hire and promote women and minorities. Some training programs were more effective than others: Voluntary programs were better than mandatory ones, and those that focused on the threat of bias and harassment lawsuits were worse than those that did not. But even the better programs led only to marginal changes. And those that were mandatory or discussed lawsuits – the vast majority of the programs the researchers examined – slightly reduced the number of women and minorities in management. Required training and legalistic training both make people resentful, the authors suggest, and likely to rebel against what they’ve heard.

What worked much better than even the best training, the researchers found, were more structural measures: minority mentoring programs, or designating an executive or a task force with specific responsibility to change promotion practices.

“You can imagine, if you’re in a meeting for two hours once a year to refresh your diversity awareness, what’s the effect of that going to be compared to being a mentor to someone?” says Dobbin.

At least some diversity consultants seem willing to accept that research finding, while still defending the role of training programs in an overall diversity policy:

Diversity trainers concede that there are poorly designed programs out there. There are also, they point out, companies that implement diversity training without much concern for whether it works, which is not a recipe for success. That doesn’t mean that well designed, conscientiously applied programs don’t work.

And diversity consultants bristle at the suggestion that they believe diversity training programs are a panacea. Properly instituting a diversity training program, many of them insist, means combining it with other, more systemic changes, including measures like those that the Kalev, Dobbin, and Kelly research found were more effective.

“If you look at just the efficacy of diversity training programs, that’s not how we look at it as a practitioner,” says Rohini Anand, global chief diversity officer at the food services giant Sodexo. “To me diversity training is one small but very necessary piece of what I need to do.”