As the federal immigration department relies more on technology to help address a growing backlog of applications, it has compiled an internal “playbook” with guidelines for what it’s looking to automate, and how it plans to prevent algorithmic discrimination.

But lawyers say the document—a copy of which The Logic has obtained—raises questions about how Immigration, Refugees and Citizenship Canada (IRCC) plans to use artificial intelligence in its modernization push.

The department is under pressure as the government sets higher immigration goals while also facing a long backlog of applications still in need of review after the COVID-19 pandemic brought immigration, and IRCC’s work, mostly to a halt. With new arrivals to Canada already accounting for almost 100 per cent of labour force growth, IRCC is being asked to not only pick the right candidates, but also to process the mountain of applications on schedule. 

It is looking to automation for help on both fronts. As The Logic first reported, some IRCC employees created a software tool called Chinook to help them deal with rising caseloads, but which sparked outcry from lawyers because it was created without legal oversight and made it difficult for them to understand, let alone challenge, the reasons IRCC denied their clients’ immigration applications. The department is seeking a replacement for Chinook, and is likely to digitize even further, budgeting $428.9 million over five years in 2021 toward replacing its central software case-management system, and using “advanced data analytics” to sort and process temporary-residence cases (though not to make or recommend denials).

The Logic obtained IRCC’s 56-page automation playbook, dated February 2021 and described as a “living document,” via an access-to-information request after spotting a photo of the document’s cover in a public government slide deck. The document, written by the digital policy team in IRCC’s strategic policy and planning branch, doesn’t identify specific courses of action, offering instead a “basis for decision-making” about implementing automation. It says the year-over-year growth in the volume of applications to immigrate, enter or stay in Canada is a primary driver for its Al adoption and that there is “an opportunity cost” to refusing to use automation, in terms of both quality and time. IRCC “should not be so cautious as to miss out on the potential” of big data, though it also should not adopt new technology for “the sake of going digital,” the document says. 

The playbook says IRCC should allow the use of black-box algorithms—which it defines as deep-learning systems or other software tools that are unpredictable, “opaque” and “working outside the scope of meaningful scrutiny and accountability”— for “business optimization” like deciding “where to locate offices or how to best distribute resources.” However, the playbook says open-source tools are the default and these black-box algorithms cannot be used as a sole determinant of who will immigrate to Canada, although technologies like facial recognition could play a role if “reliability levels” are “sufficiently high.” 

According to the playbook, the department is expected to use human rights law to determine if an algorithm would separate applicants on Charter-protected grounds and if so, whether that would be discriminatory, on top of government-wide analysis programs designed to prevent discrimination. 

One passage notes that when determining whether a task should be “shifted from humans to machines, you should ask yourself, ‘Is this a situation in which reasonable minds may differ?’ If yes, automation is ill-advised.” 

The playbook says IRCC must be transparent about AI, telling clients how their applications were processed, telling Canadians how public money was spent, and giving experts and media a basis for “informed critique.” At the same time, IRCC notes that it has an interest not to disclose too much information to prevent applicants from “gaming the system,” and that potential immigrants should not be offered a choice to opt out of automation in favour of human review because it would compromise IRCC’s processing speed and consistency, while “erroneously implying” that automated systems are inferior.

The department did not respond to The Logic’s questions about the playbook by deadline.

Petra Molnar, who previously studied Canada’s approach to automation in immigration and who reviewed the playbook at The Logic’s request, said it’s a tough balancing act for Canada to fix an immigration system that is known to be “opaque, discretionary, slow [and] laborious,” but also avoid the “egregious” way technology initiatives like Clearview AI, facial recognition at Toronto’s Pearson International Airport and predictive policing have rolled out in Canada. 

“I understand that for a country like Canada that has a vested interest in bringing people over for economic reasons …. there might be some use cases where AI, automated decision-making, can be used to make the system more efficient,” said Molnar. 

“We’ve missed the step where we’ve had a conversation as a society about whether or not we’re actually okay with automating certain aspects of a high-risk decision-making system like immigration.” 

The government said last month it will admit more than 1.3 million permanent residents over the next three years, more than it had previously projected after breaking a record of 401,000 new permanent residents in 2021. The increase comes after the pandemic’s “deep and sudden impacts” on IRCC’s operations, “much of which were paper-based,” Immigration minister Sean Fraser noted in the department’s annual report. 

Automation was a topic of debate in House of Commons immigration committee hearings last month, when Gideon Christian, who teaches about artificial intelligence and law at the University of Calgary and is president of the African Scholars Initiative, reported worsening rates of Canada’s student visa approvals for applicants in African countries. 

“Racism and discrimination is evident from a human review of this application. If we train artificial intelligence technology using this data, we’re going to have a regurgitation of that same problem, this time not by humans but by technology,” Christian said in the hearing when asked about the Chinook tool.

“The entire use of Chinook technology and artificial intelligence is embedded in secrecy and a black box…. I don’t even have access to this to be able to tell you [whether] this technology is amazing or discriminatory.” 

Lou Janssen Dangzalan, a lawyer at LJD Law whom The Logic asked to review the playbook, said that the playbook is set against a backdrop where the use of artificial intelligence or other technologies in decision-making is inevitable.

“This is something that’s going to happen, and it’s very critical at this juncture that Canada gets this right,” Janssen Dangzalan said.

To improve public confidence, the playbook recommends putting a “human in the loop” that’s accountable for automated systems, and says it’s important for officers to manually review cases alongside automated systems—and that “officers should be informed, not led to conclusions” by AI. Algorithms should be reviewed by external experts for auditing, testing and review “where possible,” the playbook adds.

Molnar, who is now based in Greece, noted that the European Union proposed regulations last year that listed immigration as one of eight “high-risk” uses of AI, sparking a public discussion that should also be considered in Canada.

“This needs to be part of a public conversation, and perhaps the government would develop some sort of risk matrix and really talk about how it’s going to make sure that true black-box AI tools will be of limited utility and will be used for really specific instances,” said Molnar.