The artificial intelligence boom presents Canada with unique opportunities and risks as we seek to benefit from a technology that could reshape how we live.

In this special series, Canada’s AI Advantage, The Logic examines how Canadian companies, investors, institutions and workers can gain from the country’s early lead in AI, even as Canada’s pioneers in the field become the world’s most powerful voices of caution.

MONTREAL — Foteini Agrafioti has interviewed a lot of machine-learning PhDs while assembling one of the larger corporate teams of AI researchers in the country. “The first question they ask you is, ‘How many GPUs do you have?’” says Agrafioti, head of RBC’s Borealis AI unit. It’s an odd inquiry to make of a potential employer. “But it’s true—it’s a scarce resource.”

To train the models underpinning artificial intelligence systems and tools, developers need “compute,” the industry term for the processing power and infrastructure through which the programs run. Those stacks include software as well as hardware like graphics processing units (GPUs) and other chips. 

Massive, multipurpose foundation models—like the large language models behind ChatGPT—require many times the compute to train that smaller ones do. Between September 2015 and February 2022, the amount they needed to get up to speed doubled about every 10 months, according to a contemporary study. While training takes a lot of compute up front, “inferencing”—an application’s output, like an answer to a query or a summary of a document—adds up over time, as the system is used over and over. 

To thrive in the AI boom, say Agrafioti and others in the sector, the country needs to ensure startups and researchers have access to the infrastructure to make those training runs, or to run applications that require lots of cloud time. Other countries are spending significant sums to build new supercomputers and shore up their hardware supply chains. “Canada needs to figure out its computational power challenge,” Agrafioti said. 

Borealis has enough GPUs for its own work, which includes AI research and product development. In July 2020, RBC announced a deal with chipmaker Nvidia and software major Red Hat for a private cloud setup. But Agrafioti—who also co-chairs the federal advisory council on AI—said she’s concerned about whether smaller firms can access “this extremely expensive resource.” 

AI startups, particularly those in the buzzy generative space, have been raising huge sums of venture capital, even as the flow of money into other tech sub-sectors remains low. But many are spending a significant share of that funding on compute; in April, partners at Silicon Valley financier Andreessen Horowitz estimated that, for several firms, it’s as high as four-fifths. 

Some deals directly link the two. Microsoft has committed over US$11 billion to OpenAI, reportedly including credits for its Azure cloud. Anthropic selected Amazon Web Services (AWS) as its primary compute provider when the tech giant announced a US$4-billion investment in the  firm. 

Toronto-headquartered generative AI startup Cohere has rejected deals with “money that needs to be recycled back into the investor,” CEO Aidan Gomez wrote in a note to staff last week. While such “arrangements of convenience” provide access to compute, companies can simply purchase it in the open market, and costs tend to drop over time, he told The Logic in January. 

Intel’s Canada country manager Denis Gaudreault has been getting a lot of calls about the semiconductor giant’s AI chips and software recently, especially since ChatGPT debuted last November. “People [are] realizing, ‘I better get on that bandwagon, ASAP.” Intel doesn’t sell or ship directly to telcos, banks or other end users in Canada. But it’s working with customers here to dispel what Gaudreault calls the “big myth” that they need to buy GPUs for every AI job.

It’s not just industry that needs compute infrastructure. Over the last two decades, researchers at Canadian universities made key breakthroughs in modern machine learning. But academia faces challenges to keep up with today’s most popular systems, said Elissa Strome, executive director for the Pan-Canadian AI Strategy at CIFAR. “We don’t have the computing facilities necessary to do generative AI on the same scale that OpenAI does.” That’s a problem, because independent researchers play a role in ensuring the technology is developed and deployed safely, she said.

Both new and established Canadian firms use the services of the very largest cloud providers, called hyperscalers, which include the U.S.-based giants AWS, Azure, and Google Cloud. All three have programs offering credits for startups and researchers and have built multiple clusters of data centres in this country, allowing clients to meet any requirements to store information locally.

But Canada needs its own compute capacity for homegrown science and products, according to Dan Desjardins, CEO of Kingston, Ont.-based Distributive. “The Americans are making all the money—they’re developing and deploying all the infrastructure,” he said. Distributive’s clients include hospitals and universities that use its software to split their AI and other workloads across the spare processing power of a network of computers, servers and data centres. 

Desjardins said the 40-person firm has raised about US$10 million to date. It is now part of a consortium trying to sell the federal government on a program to supplement Ottawa’s existing science-compute system, operated by the Digital Research Alliance of Canada.

While several nations have launched AI strategies to earn a share of the technology’s economic dividends, fewer have taken stock of the compute capacity available within their borders, according to a February 2023 paper published by the OECD. “This is akin to having a public health strategy without knowing how many hospital beds or ventilators you have,” said Celine Caira, a Canadian economist at the organization who led work on the report. 

The multilateral group has pushed policymakers to assess how much compute their countries have, how much they need, and what they can do to close any gap. The report also suggests countries consider factors like sustainability—data centres require a lot of power, which generates emissions—and security. 

Some governments have pulled out their chequebooks. Earlier this month, the U.K. announced it would spend £300 million ($506 million) on two supercomputers, which researchers can start using next summer to develop new AI applications and safety test foundational models. Countries are also offering billions to reshore production of semiconductors, critical components of compute hardware. 

Beyond the big investments, some governments are trying to make better use of existing infrastructure, Caira said, citing moves like the EU opening its high-performance computing network for science to startups. “For researchers … you have quite a lot of capacity available,” she said, noting that “many of these platforms were not necessarily available for commercial activities.”

The federal government is working on the infrastructure issue, Innovation Minister François-Philippe Champagne told The Logic earlier this month, adding that he’s discussed the subject with both startups and cloud providers. “If you want to be a leader in AI,” he said, “you need to have affordable access to that computing capacity.” 

Desjardins said securing that processing power is crucial to AI-driven innovation, whether it’s in medicine, aerospace or beyond. “To compete, you must compute.”