The rapid development of artificial intelligence (AI), particularly in recent times, means that it is increasingly influencing our day-to-day lives. Its uses are wide-ranging, from facial recognition to advertising on social media, and companies are increasingly utilising AI to enhance efficiency. In an employment context, AI is often seen as a real opportunity to remove unconscious bias and improve diversity and inclusion (for example, by optimising the recruitment process). However, can we be sure that AI is not exacerbating the very problem we are trying to solve? Recent guidance from the Equality and Human Rights Commission (EHRC) has drawn attention to the fact that there is a real risk posed by the use of AI, with biases within the systems often stemming from its use of training data. In these instances, historic biases from the humans inputting the data can lead to the AI exhibiting the same discrimination that it is designed to alleviate. One of the main ways this issue can be tackled is through equality impact assessments, which can be a great way to protect your organisation from claims and maximise the opportunities presented by AI to drive down bias.

Five reasons to carry out an equality impact assessment when using AI

(1) It helps organisations understand how their AI works

Although a general understanding of AI is becoming more common, many organisations implementing such systems are still in the dark about how they actually work. AI-based tools are often referred to as a "black box", meaning that the input and output data are known, but the algorithm used to determine the output is not. This has understandably led to distrust in the system as, if you cannot understand how a conclusion has been reached, it is impossible to know whether the decision-making process has been fair. This issue can be minimised if organisations have a good understanding of the source material used to teach the AI system and of the algorithm subsequently used to produce its results. An equality impact assessment can help an organisation to gain this understanding. Such assessments evaluate the systems in place, and consider whether any historic biases are influencing the data set used to teach the AI and, therefore, the output of the AI decision-making process. By way of example, the AI may have been taught to positively respond to language more commonly used by men when evaluating job applications, which can result in fewer job opportunities for women. Equality impact assessments assist organisations in understanding where biases and discrimination lie, so that steps can be taken to remove them.

(2) It helps organisations understand the impact of AI decisions

Equality impact assessments also help organisations to consider the risks posed by any potential faults in their AI systems and, therefore, to take steps to mitigate them. In an employment context, the legal and reputational risks posed by potential discrimination in the recruitment process can be significant. If recruitment decisions are being made based on biased data, this can open a company up to both reputational damage and discrimination claims. Additionally, it is important to consider whether data protection laws are being complied with. As is the case whenever data is being processed, organisations must consider whether data is being used fairly, lawfully and transparently, as well as whether there are any risks arising from its use.

Whilst in other contexts, such as the use of AI to optimise web searches, the risk may be lower, there may still be risks present. An equality impact assessment can assess any use of AI, whatever its purpose. Moreover, it can help identify any "human" aspects of the decision-making process, to ensure that any flaws in the algorithmic decision-making are caught and rectified by an objective decision-maker(s).

(3) It reduces the legal risk of discrimination claims

One of the key ways in which equality impact assessments can be an asset to businesses is by reducing the legal risk of discrimination claims. Under the Equality Act 2010, it is unlawful to discriminate against anyone on the basis of a protected characteristic, which includes (amongst others) age, race, sexuality and gender. However, without assessing the input data and algorithm being used by the AI, it is impossible to know whether it is utilising discriminatory criteria to make decisions. If a claim is brought, an organisation can be liable regardless of whether the discrimination was intentional or not, so it is vital to ensure that AI is helping to tackle the problem rather than create it.

(4) It reduces regulatory and reputational risk

The issues already outlined give rise to potential discrimination claims at an individual level. However, they can also lead to adverse publicity and wider claims as a result. Social media exacerbates this risk. Several sources, including the government and the EHRC, have highlighted that many institutions may not even realise that the AI they are using is flawed. It is therefore important to be vigilant and have a thorough understanding of the systems being implemented as, once criticisms are made, it can be difficult to repair any damage to your reputation. For example, in the US, a 2019 study found that facial recognition AI technology widely used by law enforcement agencies was racially biased and misidentified people of colour more often than white people. As a result, companies including three of the world's largest technology companies either paused or ended sales of their facial recognition technology to police in the US.

Where there is a larger-scale issue, there may also be regulatory issues to consider. The Secretary of State for Digital, Culture, Media and Sport has recommended that regulators consider "lighter-touch" options, such as guidance or voluntary measures, rather than sanctions, in the first instance. However, the penalties for misuse can be severe, with the European Commission recently publishing draft legislation imposing a maximum fine of €30 million or 6% of annual turnover, whichever is higher. Whilst this legislation would not form part of UK law, it has extra-territorial effect and would be relevant if an organisation uses an AI system in a way that affects people located in the EU. The level of regulatory risk will depend on the way in which the AI is being used, which is why an equality impact assessment is vital. 

(5) It increases and strengthens the opportunities for improvement

Aside from reducing the risks posed by AI systems, equality impact assessments can be a great opportunity to ensure that the maximum benefit is being derived from the use of such systems. As well as identifying any in-built biases, an equality impact assessment can look at the areas in which the AI is being used, in order to evaluate whether it is the correct fit and maximise efficiency. Moreover, an assessment can help all those involved in the decision-making process to gain a greater understanding of how the data is being processed, including with regards to IT, data protection and equality, in order to make any changes required. This allows for transparency and accountability in data processing, and enables contractual safeguards to be put in place. An equality impact assessment can also look at the lifecycle of the AI to allow for continued improvements as the use of the system adapts.

Conclusion

AI is an exciting tool that offers an opportunity to modernise recruitment processes and many other aspects of the employment landscape. However, it is vital to consider the risks it poses and ensure that all possible steps have been taken to mitigate them. An equality impact assessment is one of the most effective ways to do this. Dentons has a wealth of expertise in this area and can carry out an equality impact assessment in relation to AI usage in order to help protect businesses and create opportunities. If this is of interest to you, please do not hesitate to get in touch with our People, Reward and Mobility team.