While we all know that diverse teams have been proven to lead to greater business success, there’s no shortage of stories of diversity programs that have failed. One intriguing reason may be due to psychology and the extent to which our bias can works against diversity – even when we’re smack in the middle of a diverse team.

You’ve certainly read more than one article about ‘why diversity programs don’t work’. With so much at stake and mountains of evidence on the benefits to a company’s bottom line, why aren’t organisations making more headway when it comes to recruiting and retaining diverse teams?

The answer lies in the hidden biases at work that influence the way we feel when working in diverse teams – even when we know how good they are.

Speaking to journalist Ezra Klein on his podcast, technology anthropologist and computer scientist Danah Boyd describes how technology has exposed us to more homogenous thinking – ironically, given the opening up of communication – rather than exposing us to more diverse thinking.

In a similar way, exposure to diversity initiatives can have the same effect.

Why?

Simply put, because diverse teams make us uncomfortable.

When working in diverse teams, people believe themselves to perform worse when they are working in more uniform teams, says Boyd. “Not only that, but they also believe themselves to be less happy when they’re in diverse teams.”

She’s referencing the work of social psychologist Mahzarin Banaji who has conducted several studies about how unconscious bias shapes our perceptions of diverse workplaces, despite our awareness of evidence that proves the opposite. One example of this is her Implicit Association Test (I.A.T.), which, in one iteration, tests whether white participants are more likely to associate “good” words with white faces than with black faces (spoiler: they are, and black participants show the reverse results).

Her tests show that many of us hold onto unconscious bias, even when striving to act without prejudice.

Several other studies in recent years support the idea that homogenous teams feel easier and therefore are perceived to be more effective.

Intuitive bias

This intuitive bias, which psychologists call “the fluency heuristic”, describes how we prefer information that is processed more easily, or fluently, because we consider it “to be truer or more beautiful.” It’s why we gain greater appreciation of songs or paintings when they become familiar: because they’re more easily processed.

“As a result,” says Boyd, “when people are optimising for happiness, or when they’re optimising for ‘perceived’ success – they actually go to homogeneity.”

Boyd gives another example of human’s natural reversion to homogeneity by way of an example about the video streaming service Netflix. As the service began to transition from their original business model (which was primarily about sending people DVDs) to offering more streaming, it realised that while we aspire to want to watch a serious, award-winning movie like 12 Years a Slave, “on any given night I get home and I’m exhausted and I look at that DVD and think ‘ugh I can’t do that tonight’.” What people were instead doing was ignoring the DVD and watching the streaming option that “was lighthearted and makes you feel good in the moment.”

It’s this amazing tension, Boyd says, “because our ideal selves often want us to be reading longform, thinking deeply, really engaging with the politics of the day.” But our immediate selves unconsciously self-correct. When life gets busy, we revert to comfort.

Machine learning robots and AI were thought to be the solution to our implicit human bias. However increasingly we are recognising that AI such as machine learning algorithms that interpret speech and text, tend to adapt to reflect biased attitudes. The most well-publicised example of this is Microsoft’s ill-fated learning chatbot “Tay”, which had to be shut down only 16 hours after it launched when, mimicking language it had encountered, it went on a sexist and anti-semitic tirade on Twitter.

So, how do we get past anti-diversity bias?

The lesson here is recognising that the obstacle we face isn’t simply that hiring managers and leaders tasked with building diverse teams don’t care enough about doing so. There are biases that perpetuate even when diverse teams are in place.

Getting a diverse team together may feel like winning the battle; it’s actually the start of the war. While bringing these biases to light doesn’t solve the problem, it enables us to combat them in order to find solutions.

Enjoy full access to the online peer reviewed HR journal – Asia Pacific Journal of Human Resources, exclusively for AHRI members. Become an AHRI member now.