OTTAWA — As the federal Immigration Department struggles with what to do with scores of students who entered Canada with the help of documents they didn’t know were forgeries, it has an artificially intelligent bot going over applications in the hope of avoiding repeats.

Immigration, Refugees and Citizenship Canada (IRCC) spokesperson Sofica Lukianenko confirmed the AI-driven process is in use now, looking for patterns in previous applications that ended badly—with findings of fraud or criminality or expulsions from the country, for example—and flagging new applications with similar features.

The “integrity trends analysis tool” is described in a document called an algorithmic impact assessment, which departments have to complete for any new automated process they create to make decisions affecting members of the public.

These are like questionnaires, laying out what the tools are supposed to do and what safeguards their designers have set up.

The Immigration Department’s fraud bot “analyzes large volumes of data and extracts various risk and fraud patterns within applications containing adverse characteristics,” according to its description. When new applications match those patterns, the bot flags them to the department’s risk-assessment units.

Human staff spot-check applications for accuracy and honesty. Previously, they chose randomly or relied on their qualitative impressions of fraud trends, according to the department’s explanation of the new bot. The tool gives them analytical data to guide them.

“Examples of verification activities include authenticating proof of funds from a bank or verifying a letter of acceptance from a Designated Learning Institution,” the fraud bot’s algorithmic impact assessment says.

Those letters of acceptance are at the core of catastrophic disruptions in the lives of numerous people—as many as 700, by some accounts—who came to Canada to study, apparently without knowing they had been given bogus letters by shady immigration consultants.

In many cases, once the people were in Canada, their consultants claimed something had gone wrong with the institutions where they were supposed to study and redirected them to other schools. The students applied from within Canada, were accepted, studied and graduated. The government detected the fake letters when the students applied for permanent residency and their documents and histories were re-examined, years later. Stories of people facing expulsion as a result have dripped out through the spring.

The department has identified several dozen instances where the evidence of forged letters has been strong enough that the government has ordered people out of the country, Immigration Minister Sean Fraser said in a Parliament Hill scrum on Wednesday.

But he’s ordered a stop to those removals. In his estimate, a few hundred people might have been admitted based on fake letters, and if they had no idea they’d been scammed, it’s unfair to kick them out of the country, Fraser said.

(The AI tool was not used to detect these cases, Lukianenko wrote.)

A special task force will go over each case to try to determine whether the people involved knew their letters were bogus, Fraser said.

The department gets “many hundreds of thousands” of applications for study permits each year, Fraser said. “We have been working over the past number of months to establish a stronger system to better detect fraud when it comes to letters of acceptance,” he said.

Some of the details in the fraud tool’s documentation are worrying, said Prof. Renée Sieber and Ana Brandusescu, who study artificial intelligence policy at McGill University.

Sieber is concerned about the data used to teach it what to look for. That data would need to be “de-biased” to keep the tool from just making systemic errors faster, Sieber said.

What are some of any applicant’s key characteristics? “For IRCC, it’s going to be location, it’s going to be age, it’s going to be gender. Well, those are racist and sexist and ageist,” Sieber said.

But if you scrub those out or “fuzz it,” you risk removing characteristics that could be important for decisions like refugee determinations, she said.

The assessment of that bot says the department has processes for testing data for bias and “to manage the risk that outdated or unreliable data is used to make an automated decision,” but neither is publicly available.

The Logic asked Immigration Canada to spell out what these processes were.

Lukianenko sent a general response about the department’s determination to “align with or exceed government of Canada best practices.” The department regularly checks to see that automated tools reach the same conclusions as humans processing applications do, she wrote.

“A team of experts, including officers who process applications and privacy, legal, and policy experts, is involved throughout the process of developing and using these tools. Changes to tools that help with processing applications only happen with human oversight, testing, and approval,” Lukianenko wrote.

The assessment also says the department sought advice on the algorithm from just one external source: Statistics Canada. No human rights advocates or immigration experts.

“The fact that an external review, to this criteria, is anyone outside of that agency but still in the federal government is a weird categorization,” Brandusescu said.

Besides the fraud-detecting tool, Immigration Canada has two other new algorithms that are meant to whip through different types of applications and deliver quick approvals in easy cases.

One of the tools to speed up processing is for applications for visas and work permits for Ukrainians who want to come to Canada under the program the federal government set up for migrants fleeing the Russian invasion. The second goes over applications from people who want to be private sponsors for refugees in Canada.

The immigration system is notoriously backlogged but, Lukianenko wrote, this spring’s most recent figures showed that 60 per cent of the applications in the department’s queue were “within service standards” as of the end of April, an improvement from 56 per cent at the end of March.

That still leaves 809,000 applications waiting longer than the government’s targets.

“When these tools are able to make some positive decisions on routine applications, officers can focus on reviewing and deciding on more complex cases,” Lukianenko wrote.

Lukianenko wrote that these two processing tools are also in use, but did not answer a question about how many files they have processed, how many quick approvals they have given, or how many files the integrity tool has flagged for human checks.