Artificial Intelligence is hitting its stride – granted it’s still intelligence with a lower case “i” at this point – and its potential to fundamentally transform our economies and our day-to-day lives is just now starting to be understood, as well as felt.

Data fuels all AI solutions. That is fact, even if the definition of artificial intelligence is still up for debate (for example, is machine learning really “intelligence” if it is merely automated rules workflow processing?). Data is digested, crunched, structured, and data is spit out the other side of algorithms that serve form the foundation of AI and Machine Learning.

Because humans are the origin of both that data and those algorithms, that data and those algos are prone to one very human thing: bias.

And bias is expensive. Don’t believe me? Here’s a little proof:

On 23 April, 2013, after a fake news tweet about explosions at the White House which triggered high-frequency trading algorithms – based on sensitivity to emotional word triggers – to go into overdrive reaction about a fake terrorist attack. Capital markets that day lost $136bn in three minutes.[1]

Three minutes later, humans identified it as fake news, but the damage had already been done in the financial market. The cost to the markets may have been the result of a very simple rules based algorithm, one that, had these new high frequency trading patterns been fused with more sophisticated machine learning mechanisms, might have been avoided.

But this is where one of the fundamental questions about AI, finance, and machine learning comes into play. It is not just about incorporating smarter machine learning, nor about the most sophisticated maths being applied to assess risk profiles, market trends, or even identifying transaction anomalies that predict fraud. It’s actually about how to make strong AI without bias.

So, can we actually remove bias from AI algorithms if it is human beings who create those algorithms?

It’s not easy, especially since it’s baked in from the beginning, and that bias is reinforced every time the algorithm runs, every time a machine learns. Algorithmic bias is real, and it happens when a computer system behaves in ways that reflect the implicit human values involved in that data collection, selection, or use.

Not convinced? Need more proof?

Three Princeton University academics have shown that AI applications replicate stereotypes shown in human-generated data, and this prejudice applies to both gender and race[2]. Using the GloVe algorithm, these academics reported that machine learning “absorbs stereotyped biases” when learning language (a rule-based exercise) after running an experiment with word pairing. For example, European names were consistently associated with pleasant terms, compared to unpleasant associations common to African sounding names. Same goes for female names, which were more associated with family than career, as compared to male names. Each Machine Learning test replicated the same results as those tests completed by humans.

What the Princeton academics proved in their study is what we call Pre-Existing bias, which is a consequence of underlying social and institutional ideologies. It preserves those ‘status-quo’ biases, and replicates the bias into all possible uses of the algorithm into the future.

Another type of algo bias is technical, due to limitations of a program, computational power, design or other constraints on the system. Technical Bias happens when we attempt to formalize decisions into concrete steps on the assumption that human behaviour will correlate.

And then there’s Emergent bias, which is the result of the use and reliance on algorithms across new or unanticipated contexts. What does that mean? New forms of knowledge, such as new laws, business models, or shifting cultural norms, may be discovered without algorithms being adjusted to consider them, and BOOM, Emergent Bias happens.

But wait, there’s more!

It’s not just the algorithmic design that can be bias, it’s the data those algos run on that is fettered with bias, and this is the more pernicious threat because machine learning systems are only able to interpret the data they are trained upon.

This is especially true when it comes to Deep Learning AI. Why? Because deep learning systems are designed to learn from input data and apply learnings to other data through micro-calculations about that data. Deep learning neural networks are even more tricky, because which micro-calculations, and in what order those calculations are made, are determined by the DATA, not the algorithm.

We have yet to discuss the challenges of the data bias, so stick with me here.

Despite the vast, and I mean vast amounts of data we create every day (2.5 quintillion bytes daily, and that rate is rising every minute), how we use that data has limitations, especially structured data for structured learning.

There are four types of structured data: Open source (usually specific to a research topic), Artificial (it’s been created to use to test, most often found with image recognition data, where the images have been annotated), Web (requires a human to sort and make sense of it first in order for it to be meaningful), and Annotated (again, a human has to sort and file it for meaning).

All structured data is touched by humans. All of it. And we need structured data to give the machines context and content understanding (think SEO word search engines, or FAQ responses for chatbots). We structure data so we can apply hierarchy and organization to it, so we can set up an ontology structure that allows us to make meaning and relationships between the data for interpretation. But again, that structure is subject to qualitative categorization, which can be highly subjective, especially when that data is attached to particular social, political and legal norms. Recognize the risk of bias here?

What’s fascinating is that unstructured data can point out these biases to us – unstructured data is primarily used for pattern recognition, the unstructured bit is us asking the machine to tell us what it sees, rather than us telling it what to look for. It’s incredibly useful for pointing out what we don’t know, and it helps us address the ‘correlation does not equal causation’ conundrum. But it doesn’t help with context and meaning, we need structured data for that.

So how do we counter the very real, very pervasive risk of bias in both algorithms and data? There’s only one precaution we can take: Diversity.

Starting with the diversity of the teams building our AI systems. It’s crucial for that team to consider whether any elements of human, cultural or systems diversity have been overlooked. Unless team members represent those dimensions of diversity, it’s almost impossible to have people present to ask the necessary questions and challenge the assumptions we have around human behavior or levels of inclusion/exclusion based on stereotypes.

Diversity is equally as important in the ontology we use to structure our data, as well as what data we choose to train the machines: exclusion of certain groups from data sets that AI uses leads to inability solve problems or challenges that are outside of the data set. Exclusion is a common thing when the people choosing the data are all the same; diversity in the team selecting the data means a higher level of inclusion, and inclusion can mitigate some of the bias risk. Why? Because statistically speaking, the more unique data you accrue, the higher the probability that said data will span a more diverse range of features. And that makes for better output.

Diversity is the only insurance policy we have: diversity of teams, of data, of rigorous questioning our own assumptions. As long as humans are creating the algorithms powering AI and machine learning, and as long as humans are qualitatively structuring the data we use to train the systems, the possibility for biased outcomes exists. And we know bias is exceptionally expensive. Diversity is the only discount to that cost that exists. Diversity is the one antidote to costly algorithmic bias.

[1] Official Associated Press Twitter account @ 13.07 on 23 April, 2013: “Breaking: Two Explosions in the White House and Barack Obama is injured.”

[2] Semantics derived automatically from language corpora contain human-like biases, by Joanna Bryson, Aylin Caliskan, and Arvind Narayanan, published in Science