OTTAWA — The federal government’s rules for its own use of artificial intelligence and algorithmic tools aren’t broad enough and risk undermining public trust in the government’s ability to regulate and apply AI, internal reviewers warn in an unpublished report on its “directive on automated decision-making.”

The review, dated March 30 and obtained by The Logic through an access-to-information request, is labelled a “working draft.” Some of the elements are incomplete. But the 12-page document itself says its purpose is to identify “use cases and gaps that pose challenges to the [federal government]’s commitment to responsible AI and limit the directive’s effectiveness in achieving its objectives and expected outcomes sustainably.”

Among those challenges: the directive doesn’t cover internal-facing uses of AI, explicitly require that the models that algorithms use to reach their decisions be tested for bias, or say anything about what happens to the results of AI decisions. Longer term, the review warns that AI in policing could be a problem, and that the government should grapple with whether and how to control experiments with AI before they’re ready for prime time.

The federal Treasury Board Secretariat, which takes care of back-office functions for the government, is in charge of the rules. Spokesperson Martin Potvin said in an email to The Logic that the department “continues to explore the issues identified in the draft report, with a view to ensuring that the directive and the related guidance adapt to the changing risk environment in the federal government and meets the expectations of clients subject to automated decision-making.”

The directive for federal public servants essentially says that the more consequential the decisions made by an algorithm, the more intense the examination the algorithm has to be before anybody starts using it.

It covers algorithmic tools put to use after April 2020. The Logic reported on it last summer; at the time, just two algorithmic tools had been put through the formal assessments the directive requires, and no more have been completed since.

As it stands, the directive only applies to decisions that affect members of the public. If an AI system is only for internal use—predicting demand for fuel for fishery enforcement boats, say, or looking for irregularities in a department’s books—the directive doesn’t cover it. Nor does it apply to hiring and promotions, performance reviews or security monitoring in government facilities.

“This is proving to be a limitation that risks damaging public trust in the government’s capacity to strike an appropriate balance between innovation and responsibility,” the draft review says.

“This is something I thought was an issue right from the beginning,” said Teresa Scassa, a law professor at the University of Ottawa who specializes in information law and policy. (The review cites a paper Scassa wrote that looks at how the directive fits into the tradition of the rather technical field of administrative law.)

“I think that part of what [the review] does is just capture this huge gap that there is, which is that there are automated decision-making tools being used within government that fall outside the scope of this directive. And how does that make any sense—that there’s going to be fairness and governance for automated decision-making for external services, but not fairness and governance when they’re turned internally and inward?”

The review mentions a Globe and Mail story early this year about a move at the Department of National Defence to use two AI-driven services to help find job applicants as part of a diversity recruitment campaign.

“Due to unintended biases, high error rates and opaque outputs, the use of AI to help filter or select job candidates could risk undermining the [government]’s commitment to fair and impartial hiring processes and to a diverse, equitable and inclusive workforce,” the review says.

Though the directive implies that the models that AI tools apply should be closely examined for bias and weak spots, the review notes that nowhere does the directive specifically say so.

A classic case is using a history of sexist hiring to inadvertently teach algorithms that a company prefers to hire men. The directive focuses on making sure the data used to train algorithmic models is high quality so they don’t learn to repeat bad decision patterns. But it’s silent on the models themselves.

“The use of machine learning to evaluate candidates during online interviews or monitor the performance of an employee can lead to discriminatory outcomes and perpetuate inequalities,” the review says.

“Affect recognition,” in which computerized systems try to assess facial expressions and body language, is particularly fraught, the directive says. Trying to use computers to tell whether someone is nervous by the look on their face, for instance, is notoriously difficult—algorithms have trouble just identifying people, never mind determining what those people are feeling.

Third, the directive also doesn’t have enough to say about what happens to the outputs of algorithmic processes, according to the draft review. Should the government insist on owning the outputs of algorithms it uses, if those tools come from outside vendors?

And what if the results aren’t “fair, unbiased and broadly compliant with applicable policy and legislation”? If a hiring-recommendation AI is discovered to be giving bad advice, what happens to the “inferences, recommendations, predictions, assessments, scores, rankings analyses and so on” it might have produced along the way? Are they kept for research and the historical record, or are they destroyed?

In an annex headed “Issues for Future Consideration (TBC),” the draft review raises legal and ethical concerns about AI use in federal policing—which includes the RCMP’s serving as provincial and local law enforcement in much of the country.

In many cases, like when police use algorithms to guide their patrol plans or to try to identify suspects, those algorithms aren’t quite making decisions, but the information they feed to officers could have big effects on what the police do.

Until the public found out and privacy advocates reared up in anger, Canadian police used a face-matching algorithmic tool created by U.S.-based Clearview AI. The review points out the AI directive wouldn’t have covered that because the tool itself didn’t make any decisions.

Once again, “to the extent that algorithms are trained on historical data, their users run the risk of perpetuating past injustices and discriminatory policing practices,” the draft review says.

Scassa told The Logic the draft review’s emphasis on expanding the AI directive’s reach is a good sign that it’s “maturing” now that it’s been in effect for a while. But she also noted that only the two algorithmic tools have been put through the assessments the directive requires and the draft review doesn’t dig into that.

“There’s no data in here … about how frequently it’s been used. For what programs and what the experience was of that, and how effective [it was],” she said. “There’s a review of the text that’s in the directive, but there isn’t a review of how the directive has functioned. You know—is anyone using it?”

The directive itself says it’s supposed to be reviewed every six months. The March 30 draft came out of a review that began last October, so the Treasury Board Secretariat should now be starting the second review since the one that produced the report.