TORONTO — A midsummer sun is beating through the blinds on the bank of windows at the back of the lecture room when Sarath Chandar rises to introduce neural networks. 

The field of artificial intelligence has been aglow for months. Companies are trying to capitalize on the technology, governments to contain it, and ordinary folk to comprehend how it will impact their lives. But Chandar sees a need for much more research to achieve true intelligence artificially, and potential in the onlooking students. “We need all of your bright minds to work towards the goal,” he says.

This hot AI summer owes much to the program Chandar and his audience gathered for on a sweltering Monday in Toronto. 

For 20 years, research non-profit CIFAR has brought together some of the field’s top researchers and students for summer school. They’re here to learn about new AI techniques and applications, meet future collaborators and put themselves in the recruitment spotlight. 

Professors and pupils who passed through the program made crucial breakthroughs in deep neural networks, a methodology modelled on the human brain now in wide use. Today, they sit atop emerging business sectors and AI schools of thought. 

The program “really nurtured the early generation of deep-learning researchers,” says Chandar, a summer school alumnus himself, in an interview. “Some of them are the leaders of the field right now.”

But not all of the 190-odd graduate and doctoral students in this year’s batch know that history, even if they could name many of those names. The summer school class has evolved. This year’s, hosted by Toronto’s Vector Institute, come from backgrounds far more varied than their counterparts two decades ago. They’re pursuing applications of great ambition in ever-more fields across society and the economy, and the market for their skills is sizzling.  

“How futuristic was this school, from the early years?” asks Chandar rhetorically. The program’s originators, he says, could hardly “have even imagined that deep learning was going to have this much economic impact in 2024.” Two decades ago, AI was an intriguing scientific possibility; today, it’s a hard commercial reality. “I’m looking for the future leaders from this summer school,” says Chandar.

---

There were 28 people at the first summer school, in July 2005. They met in a windowless room in a ’60s building at the University of Toronto filled with ’90s furniture. “It was not glamorous at all,” recalls Graham Taylor, who at the time had recently returned to Canada from France for a PhD. 

U of T professor Geoffrey Hinton gave the lecture on neural networks that first year, as researchers tried to pre-train and stack them in layers so the systems could perform better on test tasks. The previous year, CIFAR had put up $2.5 million over five years for a program proposed by Hinton that would support and connect researchers in the field. It was the depth of the neural net winter, as most computer scientists pursued other AI techniques.

Summer school was an annual rallying point for the close community that still believed in the unpopular approach. In addition to the U of T lab, the program drew from the small groups around Yoshua Bengio at Université de Montréal and Yann LeCun at New York University. “You can think about them as Geoff Hinton’s buddies coming together,” says Ruslan Salakhutdinov, then a U of T doctoral student. (The Association for Computing Machinery would later name Hinton, LeCun and Bengio “fathers of the deep learning revolution,” awarding them the field’s equivalent of a Nobel Prize.)

In the mornings, professors would deliver tutorials and students would give presentations on their research. Some afternoons were devoted to coding sessions. “We’d have little hackathons, trying to test these models [and] what they can do,” says Salakhutdinov.

Participants would sometimes scoop each other. Taylor once watched a student from Bengio’s lab demonstrating a successful solution to a technical problem that Taylor had been working on for months. “There was always this friendly rivalry between U de M and U of T,” he says. They’d also help each other out. When Taylor got his first faculty job at the University of Guelph, fellow summer schooler Hugo Larochelle was about a year into his own professorship at the Université de Sherbrooke. To help Taylor secure a key federal grant, Larochelle shared his successful application to the council for reference.

At those first summer schools, what the models could do was still relatively limited. The large language models (LLMs) currently flooding social media users’ timelines and investor pitch decks were still many years and breakthroughs away. But early summer school participants went on to instrumental roles at today’s top AI developers. 

Ilya Sutskever was a major character across several summer school editions; he’s since co-founded and left OpenAI, and last month launched Safe Superintelligence, a startup taking a guardrail-focused approach to developing very advanced systems. In addition to teaching, LeCun and Salakhutdinov now work at social giant Meta, which is spearheading a movement of open-source LLMs. 

Of course, there’s more to AI than the generative form. Raquel Urtasun presented at the first summer school while getting her PhD in Switzerland; she’d later become a U of T professor and start Waabi, an autonomous vehicle firm. Andriy and Vlad—“the famous Mnih brothers,” Taylor says—did core research at DeepMind before Google bought the startup. 

“I met so many people who now have tremendous careers in AI,”  says Larochelle, now a principal scientist at Google DeepMind’s Montreal lab, noting that “creating this network was very useful as a junior researcher.” 

---

Like the field its attendees studied, AI summer school grew gradually, then suddenly. 

Kyunghyun Cho met the program’s participants on the page before he met them in person. In the early 2010s, the neural networks enthusiast cut an isolated figure at Aalto University in Finland. “There were only a small number of the groups in the world that you could read the papers from back then,” he recalls. That’s how he first got to know the “so-called Canadian mafia.” 

Soon, he was one of them. In early 2014, Cho joined Bengio’s U de M lab as a post-doctoral researcher, and shortly after he was at summer school. The program then was still only 40 or so people, crammed together over a few days. “The community was so small, it was very easy to go from being an outsider to an insider,” he says. 

Summer school produced a major work for Cho, who had brought some of his research on a new approach to language he dubbed “neural-machine translation.” (It employed an attention mechanism—deciding which words to focus on—that would later prove particularly useful for LLMs.) Summer school conversations led to a Toronto-Montreal collaboration on a model that could automatically generate captions for images. Cho’s co-authors on the much-cited July 2015 paper included Salakhutdinov and Jimmy Ba, later one of Elon Musk’s first hires at xAI.

The community kept growing. After a master’s degree at the vaunted Indian Institute of Technology Madras, Chandar planned to head to a U.S. school for a PhD. But Larochelle, whom he’d met at the flagship academic AI conference NeurIPS, helped steer him to Bengio’s lab at U de M. 

Chandar’s first summer school in August 2016 spanned seven days and drew a larger crowd. “It was already 100-plus people, so of course not everyone can present,” he recalls. But participants could sign up for a few talk slots and show posters summarizing their research. 

Chandar went on to become a professor at École Polytechnique de Montréal and faculty member at the city’s Mila AI institute. Summer school now gives him a chance to scout students. At the 2019 edition, he spotted a promising poster and brought the researcher responsible into his own just-established lab. 

The COVID-19 pandemic didn’t stop the program, which like everything else went virtual. Luz Angélica Caudillo Mata was turning from computational mathematician in academia to applied AI scientist in industry when she attended in August 2020. “The school gave me great perspectives on cutting-edge techniques and interesting questions,” says Caudillo Mata, who’s since come back to the summer school to recruit for MDA Space, where she’s now a tech lead.

Alex Cui went the following year, as he was starting his master’s at U of T under Urtasun. “Some of the research I ended up doing … was in part inspired by the talks I heard,” he says. The lectures he cites include one from Cho, now an NYU professor, on optimizing algorithms that are black boxes to their creators. 

A presentation from the AI investors at Toronto-based Radical Ventures also helped stoke his entrepreneurial interest. After a stint at Urtasun’s Waabi, Cui co-founded GPTZero, which sells tools that teachers, cybersecurity firms and others can use to detect content generated by LLMs and other generative AI systems from developers like Sutskever’s OpenAI. Salakhutdinov is a GPTZero investor and research advisor.

Like many recent participants, Cui wasn’t closely familiar with the summer school’s history and roster of alumni—including several who’ve been significant for his own career. “That’s wild,” he says.

---

All week, Toronto’s been sweltering and muggy. The sunshine—but mercifully, not the heat—fills the room lined with light wood in the 2020s U of T building where this year’s summer school is in session. 

Some 3,000 people have passed through the program over its 20 years. This year’s participants are here for reasons that would be familiar to their predecessors—to learn more about the state of the art and connect with people who could someday advance it. 

But the student body has changed. Attendees in early years were drawn from the still-small neural-net labs at a few Canadian and U.S. universities. This year, students flew in from Ethiopia and India and New Zealand and Switzerland. There are now more women, and people of colour. 

What they’re working on has also changed. When one lecturer took a poll in the room, attendees politely called out fields ranging from ecology and epidemiology to finance and physics. “A lot of them are not trying to design the next generic neural network architecture, like it was in the very early days,” says Taylor. “They’re looking to bring the advances of deep [and] reinforcement learning to [their] area of study.”

The coursework reflects that range. Introductions to neural networks and natural-language processing shared the schedule with sessions on their applications to areas like music and robotics. “We’re learning such different aspects of AI that I would have never really explored by myself,” says Aws Al Jumaily, a master’s student in computer science at Brock University, citing a class on reinforcement learning in gaming. (There was also an unscheduled session of board games, after a climate-focused speaker couldn’t make her lecture due to flight delays.)

Elizabeth Eyeson was drawn to separate talks on using AI in health (medical tools and applications) and health care (patient interaction and nursing support). “It was interesting hearing the hard problems in both of those approaches,” she says, noting, “these tools seem to work in the most ideal settings, but clearly health care is not an ideal setting.”  Eyeson, a doctoral student at UCLA, is now thinking about how she might apply her AI expertise to the challenges of electronic health records.

The technology’s place in the world has also changed since summer school’s early years. AI is the hottest thing going right now, and everyone wants in. Other tent-pole AI events like NeurIPS have turned into “big industry fairs,” according to Taylor. “You feel, ‘Oh, this conference has really changed.’” Taylor says CIFAR has managed to maintain summer school’s research and community vibe.  

Corporate giants like RBC and Roche did appear during the program, sponsoring catered lunches and recreational activities, and sending recruiters to scout potential hires at a mid-week career mixer. Credit the increased industrial interest to the viral launch of OpenAI’s ChatGPT, which produced dreams in corner offices everywhere of new efficiencies and revenue streams. Ever since, generative AI has attracted a significant—some say disproportionate—share of companies’ resources and researchers’ attention.

“We couldn’t even dream about what’s going on with the large language models five years ago, and now it’s blown everything out of the water,” says Mbithe Nzomo, a doctoral researcher at the University of Cape Town. But the summer school programmers “did a pretty good job of not buying too much into the hype,” says Nzomo, who’s applying a variety of AI techniques to personal health monitoring.

Still, the LLMs were in the room at summer school, both as tools—live-transcribing and -translating the proceedings—and as the subject matter of the lectures. U of T professor David Fleet taught a session on generative vision. (Nzomo: “Fascinating.”) Day two closed with a panel on AI safety, and day nine began with a talk on ensuring AI’s goals align with its human creators, both subjects that are newly live in ChatGPT’s aftermath. 

Remember Chandar’s call for the application of bright minds at the top? That came during his opening lecture, when he weighed in on one of AI’s hottest topics. “These LLMs are not AGI,” he said. That’s shorthand in the field for artificial general intelligence, a theoretical stage where a machine can match a human at learning, reasoning and adapting. 

Today’s LLMs are “still very dumb models,” Chandar said, incapable of generalizing their knowledge to scenarios they haven’t previously seen. He figures it could take a century to achieve “a truly intelligent system,” and wants more smart people working to get there.

But for chance, the students would have heard a very different message. Chandar was a late substitute for Hinton, who was originally scheduled to teach the introduction to neural networks. In the wake of ChatGPT, Hinton revised his timeline for AGI to as little as five years. The AI pioneer is one of several researchers who now argue smarter-than-human systems could soon pose an existential threat to Homo sapiens. 

Wherever they land on those questions, many summer school students are aware there’s a lot of hot air in their field right now. Tech operates in cycles, Eyeson observes. Rather than getting wrapped up in the technology hype, she’s focusing on the health-care problems she’s trying to solve. AI may be the hottest thing in the world right now, she notes, but “winters have happened before.”

---

In the past, whole schools of Canadian AI researchers and students have decamped south to U.S. universities or Silicon Valley giants. The current boom has sparked renewed fears of brain drain, and of foreign firms profiting from discoveries made here with public funding. 

But CIFAR’s summer school is a not-so-secret, immersive ad for Canada’s AI ecosystem. Over the 10-day program, staff dispensed laptop stickers reading, “The world needs more Canadian AI.” (The French version: IA canadienne en service du monde.) “You are now officially members of the Canadian AI community,” Elissa Strome, executive director of the Pan-Canadian AI Strategy, told students at an event on the second-last day. 

At least one is looking to extend that membership. Nzomo made the long trip back to Cape Town last week, but her summer school experience convinced her that Canada could be the place to look for a post-doc placement. She’s now considering applying to fellowships at Vector and U of T. 

At summer school, “there was a lot of emphasis about the strides that Canada is making towards AI and how it’s really investing in this,” Nzomo says. She sees real potential to apply her work to the health system here, and an expanded network of friends and colleagues with whom to do it. “AI is such a small space,” she says.