Do machines think differently?

Table of Contents:

Instead of attempting to address these prejudicial stereotypes through some form of intervention in the hope of reducing bias against specific human groups, we could instead bypass human judgement altogether and rely instead on technology to make our assessments for us. One could, in fact, look upon this as a kind of experiment testing the hypothesis that human beings' thinking is ‘faulty’ in some way - we have a defective way of making assumptions about people. One possible explanation for this, as pointed out above, is that people base their judgements of other groups on what they see as specific attributes of the ‘out-group’ - ‘they are not like us’. Whereas we would explain our own behaviour in terms of the context in which the behaviour took place ('I dropped the dish because I was in too much of a hurry’), we tend to often attribute the behaviour of others to some internal trait (‘he dropped the dish because he is clumsy’). This is known as the fundamental attribution error. An extension of this leads to people assuming that all members of a particular group share the same attributes: ‘women have no sense of direction', ‘Americans are obsessed with guns', and ‘people who support the Palestinians are anti-Semitic’. While this kind of error might lead to how the behaviour of particular ethnic, racial, religious, etc., groups are interpreted, there still needs to be some original source for the biased thinking in the first place.

An artificial intelligence (Al) system does not have the same cognitive system as human beings and should, therefore, be devoid of irrational prejudices and attributional biases and should rely instead on the raw data presented to it. If it is the case that humans cannot avoid biases, we should expect an Al system to make judgements free of such prejudice or bias. Typically, the form that many Al systems have used is based on a neural net where the system is trained on examples with feedback from any errors it makes. This kind of system has been used over many years in medicine. To pick but one example, doctors at Moorfields Eye Hospital in London trained an Al system to recommend treatments for 50 eye diseases. Whereas eye doctors were in agreement over 64% of the cases they viewed, the Al had 94% accuracy. If a computer system is capable of this kind of accuracy after training, the view was that an Al used, say, as a recruitment tool, would make better assessments of interview candidates than human assessors with their unconscious biases. We, therefore, have the basis of a potential experiment where the results of human assessors can be compared to the results of a computer-based system.

When Amazon piloted such a technological system as part of the recruitment process for new staff to avoid potential gender and racial stereotyping, they found that the system was dismissing female candidates for no immediately obvious reason. On examination, what had happened was that the Al was trained on past recruitment decisions that were biased in favour of men. Hence, it continued to do the same thing. Technicians then adjusted the system so that it ignored words that specified a particular gender in CVs and resumes, such as ‘woman’ or ‘male’. However, it continued to reject female applicants based on ‘implicitly gendered words’ such as ‘captured’ or ‘executed’ - words that male applicants apparently used more often than females. The Al turned out to be as biased as the humans.

There are, indeed, many examples of Al systems showing systematic biases in judgement and decision-making. In the US, parts of the justice system use Al algorithms to assess the likelihood of criminals to reoffend. One system, called the Correctional Offender Management Profiling for Alternative Sanctions was analyzed by Larson, Mattu, Kirchner, & Angwin (2016). They found that it incorrectly identified black defendants as being at higher risk of reoffending than white defendants, and white defendants as being of low risk.

Bolukbasi, Chang, Zou, Saligrama, & Kalai (2016) found that Google News articles contained sexist ‘word embeddings' where words like ‘receptionist’ were linked to ‘female’. They provide a list of ‘extreme she occupations’ and ‘extreme he occupations’ that allowed the generation of analogies called proportional analogies such as she:he:: queen:king (she is to he as queen is to king). This represents an appropriate analogy. They used a machine learning algorithm to generate the endings of proportional analogies relating to jobs - she:he::?:?. When the system ran, it generated not only appropriate analogies such as the one above but inappropriate ones he:-she::computer programmerhomemaker.

‘The analogies generated from these embeddings spell out the bias implicit in the data on which they were trained. Hence, word embeddings may serve as a means to extract implicit gender associations from a large text corpus similar to how Implicit Association Tests [...] detect automatic gender associations possessed by people, which often do not align with self-reports' (Bolukbasi et al., 2016, p. 3).

Similarly, Kay, Matuszek, & Munson (2015) found gender biases in image searches for particular occupations. Also, in 2015, a search of images in Google for Chief Executive Officer (CEO) generated photographs of more male CEOs than female CEOs greater than the proportions in the general population, 11% vs 27% (Cossins, 2018). A cursory search by me for images of CEOs in mid 2019 showed a continuing similar disparity.

So, if humans have implicit biases, computer systems appear to have them too. The results of this virtual experiment show no differences between humans and other information processing systems (IPS). So, how do we now interpret the results given the original hypothesis that humans are somehow ‘faulty' in their thinking? One view is that the biases in Al algorithms reflects human biases since the data they are trained on is biased. Hence, Richard Soccer on 17 January 2019 at the World Economic forum argued that ‘Unmanaged Al is a mirror for human bias' (https://www.weforum.org/agenda/2019/01/ai-isn-t-dangerous-but-human-bias-is/). However, Karen Hao in MIT Technology Review (Hao, 2019b) points out that ‘bias can creep in long before the data is collected' and at any stage: the coding stage, the collection stage, or the selection stage. In a later article she adds: ‘Tech companies are built—and tech products are designed— with a “fantasy belief’’ that they exist independently of the sexism, racism, and societal context around them' (Hao, 2019a).

Thus, we have a chicken and egg situation. The biases come from somewhere but both Al systems and human beings are influenced by them. IPSs other than human beings are generating bias from the data they are trained on. Humans are also IPSs, so presumably we are doing the same thing. That is, our biases are due to being trained on the data through the environment and sociocultural influences we are exposed to. The environment, including the social environment, influences human activity and ways of thinking. At the same time, human activity and thought have an effect on the environment. We can therefore end up with a positive feedback loop whereby a particular opinion (or ‘meme' in the original Dawkins (1976) sense), for example, is noticed and shared which causes it to be noticed by more people and shared, and so on. This way a particular political or social view can gain currency, and we can track the rise of social movements as they increase in influence. These can be either positive or negative. Historically, there has been the rise of Nazism in Germany or Mao Zedong in China, the abolitionists in the UK and the US, the suffragette movement, white supremacist terrorism, the MeToo movement, and so on. These days such feedback loops are amplified by the ‘echo chamber' provided by social media, but more importantly they show that aspects of our culture can be changed. Our thinking is the product of the culture we are immersed in. We learn about ethics, morality, technology, stereotypes, and all the rest whether we want to or not. According to Hinton (2017a, 2017b), if it is our determination to counter implicit sexism, racism, elitism, etc. and, given it is difficult to change our minds, we should concentrate on changing our culture since the bias is in the data.

Summary

We often rely on stereotypes to make judgements about people from a particular social group since they are deemed to inherit the attributes of that group. Hasty induction can mean that the attributes of an individual from an unfamiliar group is assumed to be representative of the group. Such stereotypes can lead to potential biases against people’s ethnicity, gender, accent, class, religion, university education, and so on.

The IAT has been used to elicit unconscious bias and diversity training has been established by firms and institutions to attempt to counter or make people aware of such implicit bias.

Stereotypes and associated biases are deemed by many to arise from normal categorization processes. According to some researchers, this can cause diversity training to be ineffective or short lived. A variety of bias reduction strategies may work by catering for individual differences among people. Collective blame is often ascribed to an out-group due to the actions of a small minority and can be reduced by pointing out the hypocrisy and inconsistency involved, as collective blame is not directed at the in-group.

Al systems have been found to exhibit bias as they have been trained on data produced by people. It can be argued that people also are ‘trained’ on the same data. Rather than fixing the cognitive system, it would be more effective to change the culture that gives rise to the bias in the first place.

Do machines think differently?

Summary

Suggested Further Reading