TORONTO — OpenAI is slightly expanding parental controls in ChatGPT and touting it as an educational tool for teens, as policymakers around the world grapple with how and when kids should be allowed to use AI.

The new features build on the San Francisco-based firm’s September launch of a system that lets parents link their accounts to those of their children to set time limits, turn off image-generation functionality and alert them if “something may be seriously wrong.” Now, they’ll get those notifications in cases where OpenAI deactivates their child’s account for making violent threats. Parents still won’t be able to see the actual contents of the chats, which the firm says is to preserve user privacy. 

OpenAI is also adding more education features. Starting in January, the firm’s technology began trying to assess whether users are minors, and if so, to prevent them from seeing sensitive content that contains or promotes graphic violence, self-harm or sexual role-playing. 

For accounts tagged as belonging to teens, ChatGPT will now by default suggest learning-focused prompts for tasks it can help with, like making edits on an essay or drafting study materials like flashcards. And it will remind heavy users to take breaks more often.

The firm says most teens are using the tools to learn, access information, develop skills or make themselves more productive.

Parents can also now remotely turn on the tool’s “study mode,” which walks users through problems step by step rather than simply generating the answers. OpenAI said it used input from educators and learning experts to design the system. Teachers have expressed concern that students’ use of AI may prevent them from learning critical thinking, and make them reliant on the technology.

Policymakers in Canada and other countries have been applying more scrutiny and rules to AI developers following a series of incidents involving young people’s interactions with chatbots. 

In February, the firm promised to update its policies around alerting authorities of potential risks of harm, after The Wall Street Journal reported that it had not told police about disturbing conversations last year between ChatGPT and the teenage perpetrator of the Tumbler Ridge mass shooting. 

OpenAI is also facing lawsuits from parents alleging ChatGPT contributed to their children’s suicides, including a New Brunswick woman who’s suing in California court over the death of her 24-year-old daughter.

The company’s education-focused message comes as it faces the prospect of child chatbot bans. In a blog post published Wednesday, the firm claimed keeping teens from using AI would leave them “less prepared to use one of the defining technologies of their time.”

Manitoba Premier Wab Kinew was the first to move on a chatbot ban, in April. The province’s measure would apply to under-16s. Kinew cited the risk of mental health impacts, although he acknowledged the technology may have educational benefits.

Last month, the federal government proposed a new Digital Safety Act that would ban social media—but not chatbots—for children under 16. The law would impose on AI companies the duty to act responsibly and protect children. It requires developers that offer “public-facing conversational chatbots that can mimic human-like relationships” to ensure their products don’t communicate harmful content, and to have protocols for intervening in crisis situations. Firms that break the law, if it passes, would face audits and fines from a new regulator of up to $10 million or three per cent of global revenues from a new regulator.

Ottawa’s proposed Protecting Privacy and Consumer Data Act, meanwhile, requires companies to take extra care with the private information of children. 

OpenAI spokesperson Jamie Radice said the firm supports “Canada’s goal of protecting young people online,” and is tracking the Digital Safety Act. But she said the updates announced Thursday are part of its broader approach to safety, and expand on “ongoing work to provide age-appropriate experiences, stronger safeguards and practical tools that help families guide how teens use AI.” 

The company said it could not provide data on what share of minor-tagged accounts are linked to parent accounts.

Governments in Australia, the U.K. and the U.S. have launched or proposed partial chatbot ban or limits for minors.

Lawyers representing the families of affected teens criticized the original parental controls as insufficient, noting they require that families already know their kids are using ChatGPT.