Is your AI Program Discriminating Against Job Applicants?
Some 83% of employers, including 99% of Fortune 500 companies[1], now use some form of automated tool as part of their hiring process, said the Equal Employment Opportunity Commission’s chair Charlotte Burrows at a January 31, 2023, hearing titled “Navigating Employment Discrimination in AI and Automated Systems: A New Civil Rights Frontier[2] ” part of a larger agency initiative examining how technology is used to recruit and hire people.
Among the examples given of popular work-related AI tools were resume scanners, employee monitoring software that ranks workers based on keystrokes, game-like online tests to assess job skills, and video interviewing software that measures a person’s speech patterns or facial expressions.
Generally, AI systems work based on what is programmed into their data set. This becomes an issue when there is racial bias in the data set being programmed into AI. For example, research done by Joy Buolamwini found that self-driving cars were less likely to pick up on individuals with darker skin because of the lack of diversity in the data (“pale male data”) they were trained on.
Then there was the experiment with a once innocent AI chatbot unveiled by Microsoft in 2016 named “Tay”. Microsoft described it as an experiment in “conversational understanding.” The more you chat with Tay, said Microsoft, the smarter it gets, learning to engage people through “casual and playful conversation.”. Tay went from tweeting about National Puppy Day to supporting Hitler, being anti-women’s rights, and disputing 9/11 in less than 24 hours. John Oliver joked, “meaning she completed the entire life cycle of your friends on Facebook in just a fraction of the time”.
In selecting candidates for employment depending on the data set the AI software is programmed with, candidates may be filtered out based on who may not otherwise be filtered out if a human was looking at the employment application. For example, an AI bot may weed out candidates based on status which may rule out international candidates. There was a study was done where it was found that an AI bot filtered out female job applicants as the data set ruled out women’s colleges. There was another study done where an AI bot only considered applicants from universities in the United States because it was not programmed to consider applicants from foreign universities. Some AI software eliminates zip codes which are meant to filter out-of-state applicants but may by default eliminate someone for example applying for a position in New York outside of Manhattan from a demographically diverse zip code such as Queens or the Bronx. In guidance issued by the U.S. Justice Department and the Equal Employment Opportunity Commission, employers were warned that AI has the potential to discriminate against women and people with disabilities as it filters out gaps in the resumes of people returning to the workforce after maternity leave or individuals with disabilities returning to the workforce after recovering from medical conditions due to gaps in resumes.
The issue with learning where the racist and discriminatory policies came from on AI requires taking a hard look at how and why AI filters out the candidates it does. New York City has taken a huge leap toward this. Starting on July 5, 2023, Local Law 144 requires companies hiring employees in New York City to conduct independent bias audits of tools used to automate hiring or review/promotion processes.
Although Local Law 144 is a step in the right direction, the use of AI can potentially compound the longstanding discrimination that underprivileged and disenfranchised job seekers in our society face if not scrutinized and regulated.
My office is available to assist in working with your human resources team and management personnel to ensure that your AI software is in compliance with Local Law 144. You can also subscribe to this blog to receive email alerts when new posts go up.