OTTAWA — Three years after scorching criticisms of its experiments with using bots to help process immigration applications, the federal government is using algorithmic filters to help determine whether the spouses of Canadian residents are eligible to come here.

The new bot, in use since April, is the first major member of a new generation of “automated decision-making” tools to go through a formal federal assessment meant to make sure it has safeguards proportionate to the impact of its decisions.

For the new immigration algorithm, those are considered significant, perpetual and reliant on personal information—all red flags.

Don’t be alarmed, said Benoît Deshaies, the acting director of data and artificial intelligence at the Treasury Board Secretariat, the federal body that has ordered the assessments.

“You hear, ‘Oh, [Immigration] is using AI to decide on visa applicants.’ Well, that can sound scary, right? But when you get down to it, you understand that the automation is just supporting one of the many, many decisions that are part of that, that the final case is decided by an officer,” he said in an interview with The Logic before this week’s federal election campaign began.

One much simpler bot has been through the new assessment before. It looks over requests under access-to-information law and points out related material that’s already been released, which might be what the requester wants and be much quicker to get.

These examinations have been required—and required to be made public—by the Treasury Board’s “directive on automated decision making,” which was published in 2019, took effect in April 2020, then sat little-used through the COVID-19 pandemic.

Now, Deshaies said to expect a spate of the assessments as the public service emerges from pandemic panic mode.

The Immigration Department’s use of algorithms to speed its processing of applications was what prompted the creation of the directive in the first place in early 2018, said Ashley Casovan, who was instrumental in producing it as the director of data architecture and innovation at the Treasury Board Secretariat.

“Some high-level examples were coming out, in particular from Immigration and Refugees Canada, which was looking at piloting an automated decision-making system for expediting visas for particular students,” she said. “In this case, it was students coming from China, and because there were lots of them, they wanted to really reduce the backlog that they had.”

The idea was, and is, to accelerate simple cases, said Immigration spokesperson Jeffrey MacDonald, in an email to The Logic.

“The project’s goal is to help officers to identify applications that are routine and straightforward for faster processing and to triage files that are more complex for a more thorough review,” he wrote. 

Algorithms never reject applicants, or recommend they be rejected, MacDonald emphasized. They just pass along the applications that tick all the simple boxes.

The Citizen Lab at the University of Toronto produced a high-profile report on the potentially extreme dangers of letting AIs make determinations, titled “Bots at the Gate.” They might not have been doing so in 2018, but the prospect was there.

“The nuanced and complex nature of many refugee and immigration claims may be lost on these technologies,” it warned, “leading to serious breaches of internationally and domestically protected human rights, in the form of bias, discrimination, privacy breaches, due process and procedural fairness issues, among others. These systems will have life-and-death ramifications for ordinary people, many of whom are fleeing for their lives.”

The government needed a “binding, government-wide standard or directive for the use of automated decision systems,” the report said, one that would include limits on their use and requirements that the public be told just how they worked, right down to the source code.

Casovan—who has since left the government and is the executive director of the fledgling Responsible AI Institute—said the public service was doubly spooked at the time by the federal government’s massive overhaul of its systems for paying workers. That project took a mishmash of legacy software and complex rules and combined them all into one central system called Phoenix, which didn’t work.

Throw in international efforts at the G7—promoted by Canada and France, which led the group in successive years—to guide the use of artificial intelligence in governments, and there were a lot of reasons to try to get a handle on how algorithms were already being used and set some rules, Casovan said. 

“We had the opportunity then to create policies that really looked to oversee the management of the government’s use of automated decision-making systems,” she said. “To put guardrails around the experimentation and use of these systems, especially where we had limited knowledge—especially if there were going to be learning components in them—how they were going to behave and, ultimately, the impact to Canadians.”

The thrust of the directive that resulted is that when an algorithm is going to make or assist with decisions that affect people outside the government, the people designing it need to subject their work to scrutiny proportional to the decisions’ impact.

Deshaies offered up as a Level 1 decision whether someone is eligible for a $3 rebate on an energy-efficient lightbulb.

“At levels 3 and 4, it could be things like getting a visa for coming to Canada or imprisonment decisions, things of that nature,” he said.

High-impact, complex algorithms would demand intense scrutiny of their inputs, processes and outputs, and reviews by multiple experts, including from outside the government.

Deshaies recently wrote a plain-language summary of the directive for the government’s data scientists, with a cautionary example from Amazon, which stopped using a learning algorithm to screen job applicants after discovering it was favouring men. 

Why? The data from previous applications that had been used to train it favoured men, so the algorithm learned that Amazon wasn’t keen on applicants with, for instance, degrees from women’s universities and colleges. Rather than delivering blind justice, Amazon’s AI taught itself how to replicate failure.

Deshaies said breaking complex processes into distinct steps is one solution.

“For the time being, these systems tend to support decision making, as opposed to fully automating,” he said. 

The federal Conservatives’ election platform promises to use “technology to speed application vetting by immigration officers,” so this approach has support on both sides of the House of Commons.

More than a year after the directive on automated decision-making took effect, Casovan said she’s disappointed so few cases have made it through the assessment process. Which she acknowledged can be intimidating, despite her efforts to make it less burdensome for simple applications.

“I think that there’s a lot that can be improved. And that’s kind of what I’m working on right now. But the intent and the concepts are there, and I think that if they had been working on this over the last two years, they would have much better products in terms of best practices and how to implement these things,” she said.

Deshaies, who’s in the thick of the work, said Canada still is a leader.

“We were one of the first adopting mandatory policy instruments—a lot of nations have issued principles, things like that. So I think we do have some leadership in that space,” he said. “The systems have taken time to develop. Now that they’re being developed, they’re reaching production, we’ll see evidence of compliance.”