TORONTO — The research wing of the new Canadian AI Safety Institute (CAISI) will focus on trying to make AI technology more trustworthy, as a major international report led by prominent Montreal researcher Yoshua Bengio warned that rapid advances are also adding to the risks it could pose.

The program will back year-long projects that tackle issues like identifying AI-generated content and the use of machine-made data to train large language models (LLMs), as well as privacy concerns in applications like health care. The Canadian Institute for Advanced Research (CIFAR), which is running the research effort, will start taking applications on Thursday to fund as many as six projects with up to $100,000 each.

“We could benefit a lot from more communication and education around the capabilities of AI systems, so that people are more comfortable as they interact with them,” said Nicholas Papernot, a University of Toronto professor and the program’s co-director at CIFAR, which has $27 million in federal funding for the initiative. Ottawa launched CAISI in November, following similar centres in the U.S. and U.K.

Papernot said the program will try to answer some of the questions in Bengio’s International AI Safety Report, published Wednesday ahead of the AI Action Summit in Paris next month. The 298-page document, commissioned by the U.K. government in fall 2023, tracks the capabilities and drawbacks of general-purpose AI, systems that can be put to a variety of uses including powering automated assistants as well as generating text, images and video.

In just a few years, LLMs have gone from producing mostly incoherent sentences to solving scientific problems and writing useful code, the state-of-the-science report said. Improvements on technical tasks and tests are partly due to a technique called “chain of thought,” in which the system breaks issues down into steps like a human might when puzzling something over. For example, OpenAI’s o1 and DeepSeek’s R1—which both created buzz in the AI ecosystem—do this. 

It’s not clear how much even the most advanced systems are actually reasoning yet, the report noted. But it cautioned that more logical models might also be more useful to bad actors, helping them find cybersecurity vulnerabilities or develop diseases. Researchers worried about the technology’s existential threats have raised the possibility of malicious humans minting viruses or weapons with AI assistance, although a literature review Cohere for AI published last month concluded that LLMs don’t pose immediate biorisks.

The report also cites the emergence of AI agents, which can go out and accomplish tasks without their users watching over them. Tech firms are selling businesses automated tools for functions like customer service and research, and promising consumers assistants that can handle bookings and online shopping. But the report warns that agents could autonomously do things they shouldn’t, or be hijacked by attackers.

AI systems must be aligned with the values of their human users and developers, said Catherine Régis, the program’s other co-director and a law professor at Université de Montréal. 

In addition to risks posed by new applications, the study also cites long-standing concerns about AI, including producing discriminatory decisions because of inbuilt biases of race, gender and other identity factors; violating users’ privacy; infringing on artists’ rights to their work; and putting people out of work.

Wednesday’s report expands on an interim document published last May. Bengio, a Université de Montréal professor and scientific director of the Mila AI institute, oversaw the writing with input from 96 researchers and government officials. The report is meant to help policymakers that are looking to regulate AI, but doesn’t make specific recommendations.

One major challenge it identifies is that the experts don’t agree on the technology’s downsides, or how new advances will play out. The report claims developers don’t really know why AI models do the things they do, and that there aren’t yet good ways to monitor and measure risks.

The CAISI hopes to fill some of those gaps. “We really need to have solid research to guide policy,” Régis said. 

In addition to the grants launching Thursday, it will support networks of researchers working on longer-term problems like how to share information about AI systems’ vulnerabilities between developers, users and regulators so that they can be patched. That part of the program will open in late February, and provide up to $700,000 per project over two or three years.