Confronting the inherent bias of artificial intelligence

Seven years ago, artificial intelligence (AI) generated a public relations nightmare for Amazon. The tech giant’s AI recruiting tool, trained on historical hiring data, was systematically discriminating against female candidates.

While Amazon quickly abandoned the project, the World Economic Forum recently reported that 88 per cent of companies now use some form of AI for initial candidate screening. With global investments in AI expected to reach $232 billion by next year, addressing the critical issue of equity, diversity and inclusion (EDI) bias is more relevant than ever.

That’s where Maryam Ghasemaghaei’s research comes in.

A timely investigation

As the new Canada Research Chair in EDI Issues in Artificial Intelligence, Ghasemaghaei is exploring the phenomenon of AI bias and its impact on organizations, individuals and society. The research also focuses on developing effective strategies to mitigate these biases and promote EDI.

Ghasemaghaei’s research is structured around three interrelated projects.

• The Impact of EDI Issues in AI on Organizational Decision-making will examine how biased algorithms lead to unfair firm decisions and explore solutions like transparent AI to make fairer decisions.
• The Impact of EDI Issues in AI on Firm Employees will focus on the effects of biased algorithms on employees. It will aim to understand how such biases influence job dissatisfaction, stress and perceptions of inequality and how unbiased AI can improve employee attitudes and behaviours.
• The Impact of EDI Issues in AI on Society will investigate how biased AI algorithms reinforce social biases, affecting wealth disparities and stereotype beliefs. It will seek to develop a framework for using unbiased AI to address existing societal EDI issues.

Data, diversity and transparency

Numerous factors, Ghasemaghaei says, are central to AI’s bias, and organizations must employ a well-rounded approach to address them.

To start, AI uses historical data as input. If, for example, a bank employs AI in its loan process and its lending practices have historically discriminated against women or specific ethnic backgrounds or races, this bias will persist. While some organizations believe this issue can be resolved by providing AI models with more data, it only perpetuates a cycle, says Ghasemaghaei, if the data is incomplete, inconsistent or includes pre-existing biases towards specific criteria.

“There are a lot of consequences of having low-level quality data,” she says. “Statistics show that data analysts usually need to spend about 80 per cent of the time just cleaning the data before they start analyzing it. So, an organization must invest in resources and staff who can correctly analyze the massive volumes of data coming in.”

Other issues involve objectivity and transparency. Since humans design and train AI models, companies wanting to mitigate bias must build diverse teams and include employees with various expertise. For example, organizations should bring together cross-functional teams of professionals who understand the technical aspect of data and engineering AI models in addition to experts in social sciences who can provide different perspectives and insights on how to mitigate biases.

Additionally, transparency about how AI achieves specific outcomes is critical.

Frameworks and the future

What all these issues have in common, says Ghasemaghaei, is Canada’s lack of formal AI regulation. In 2023, the federal government announced a Voluntary Code of Conduct on the Responsible Development and Management of Advanced Generative AI Systems. While it aims to “provide Canadian companies with common standards and enables them to demonstrate, voluntarily, that they are developing and using generative AI systems responsibly,” it is not an official policy.

“There is a proposed Artificial Intelligence and Data Act (AIDA), but we don’t have specific policies or guidelines about how organizations can develop AI that does not have biases and ethical issues,” she says. “The government needs to collaborate with industry and academia to ensure it understands how to provide specific guidelines for companies when they want to develop AI or any other advanced technology. This is really important because it will standardize fairness.”

As organizations await a government framework, they must consider the long-term outlook of using AI to make decisions, Ghasemaghaei says. She warns that companies may be short-sighted in their rush to gain a competitive advantage through AI.

“I talked to some managers, and they said they don’t care if AI provides unethical decisions,” Ghasemaghaei says. “They just think about the profits. So, it may look good in the short term, but how about in the long term, when everybody is so dissatisfied with the results? When employees start going to other companies as they find that the company they work for is unethical?